Jul 17 10:50:38.175156 ip-10-0-143-42 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 17 10:50:38.175169 ip-10-0-143-42 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 17 10:50:38.175178 ip-10-0-143-42 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 17 10:50:38.175477 ip-10-0-143-42 systemd[1]: Failed to start Kubernetes Kubelet. Jul 17 10:50:48.177355 ip-10-0-143-42 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 17 10:50:48.177368 ip-10-0-143-42 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9e9f6b3c04874d4e8f00b04a73fdfcfc -- Jul 17 10:52:14.667783 ip-10-0-143-42 systemd[1]: Starting Kubernetes Kubelet... Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.127444 2551 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136609 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136630 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136633 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136637 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136641 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136644 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136646 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 10:52:15.424470 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136650 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136653 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136656 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136659 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136662 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136664 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136667 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136669 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136672 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136676 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136679 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136681 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136684 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136686 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136689 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136692 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136694 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136697 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136699 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 10:52:15.436611 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136704 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 10:52:15.212611 ip-10-0-143-42 systemd[1]: Started Kubernetes Kubelet. Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136709 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136712 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136716 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136718 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136721 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136724 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136726 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136729 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136732 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136734 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136737 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136739 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136742 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136745 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136748 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136751 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136754 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136756 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136759 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 10:52:15.438690 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136762 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136764 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136767 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136770 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136773 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136777 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136780 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136783 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136785 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136790 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136794 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136797 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136800 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136803 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136806 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136808 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136811 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136814 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136816 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 10:52:15.441129 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136818 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136821 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136824 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136827 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136829 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136833 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136836 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136839 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136841 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136844 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136846 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136851 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136854 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136857 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136860 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136863 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136866 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136868 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136872 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136875 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 10:52:15.453741 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136878 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136881 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136883 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136886 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136888 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136891 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136894 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136896 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136899 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136901 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136904 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136906 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136909 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.136911 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137683 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137690 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137693 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137696 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137699 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137702 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 10:52:15.454523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137705 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137708 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137711 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137715 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137718 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137721 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137723 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137726 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137729 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137731 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137736 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137739 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137742 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137744 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137747 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137750 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137753 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137755 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137758 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 10:52:15.456770 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137761 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137764 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137766 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137769 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137771 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137774 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137777 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137779 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137781 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137784 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137786 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137789 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137792 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137794 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137797 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137799 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137803 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137806 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137811 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137814 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 10:52:15.459148 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137816 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137819 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137821 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137824 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137827 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137830 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137833 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137836 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137838 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137842 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137846 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137849 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137852 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137855 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137857 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137861 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137864 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137866 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137869 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 10:52:15.461314 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137871 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137874 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137877 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137879 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137882 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137884 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137887 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137889 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137892 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137895 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137898 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137901 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137905 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137909 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137912 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137917 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137920 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137923 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137925 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 10:52:15.463048 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137929 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137932 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137934 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137936 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137939 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137942 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137944 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137947 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137949 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137951 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137954 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137957 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137959 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137962 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137964 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.137967 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138419 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138425 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138428 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138431 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 10:52:15.464265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138434 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138437 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138442 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138444 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138449 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138452 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138454 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138457 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138460 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138462 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138465 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138468 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138471 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138474 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138477 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138479 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138483 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138486 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138489 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 10:52:15.465703 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138492 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138514 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138518 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138523 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138525 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138528 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138531 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138533 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138536 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138539 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138541 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138544 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138546 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138549 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138552 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138556 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138558 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138562 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138564 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 10:52:15.467090 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138567 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138569 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138572 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138574 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138577 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138580 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138582 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138586 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138589 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138591 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138594 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138597 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138600 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138602 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138605 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138607 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138610 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138612 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138615 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138617 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 10:52:15.468135 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138620 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138622 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138625 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138627 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138630 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138632 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138635 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138637 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138641 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138645 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138650 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138653 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138655 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138658 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138660 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138663 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138666 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138668 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138671 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138673 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 10:52:15.469033 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138676 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138679 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138681 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138684 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138686 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138688 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138691 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138693 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138696 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138698 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138702 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138705 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138708 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138710 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138714 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138717 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.138720 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138790 2551 flags.go:64] FLAG: --address="0.0.0.0" Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138797 2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138804 2551 flags.go:64] FLAG: --anonymous-auth="true" Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138810 2551 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 17 10:52:15.470127 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138815 2551 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138820 2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138824 2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138828 2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138831 2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138835 2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138838 2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138842 2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138845 2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138848 2551 flags.go:64] FLAG: --cgroup-root="" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138851 2551 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138854 2551 flags.go:64] FLAG: --client-ca-file="" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138857 2551 flags.go:64] FLAG: --cloud-provider="external" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138860 2551 flags.go:64] FLAG: --cluster-dns="[]" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138865 2551 flags.go:64] FLAG: --cluster-domain="" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138868 2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138871 2551 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138874 2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138877 2551 flags.go:64] FLAG: --container-log-max-files="5" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138881 2551 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138884 2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138888 2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138891 2551 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138894 2551 flags.go:64] FLAG: --contention-profiling="false" Jul 17 10:52:15.471156 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138897 2551 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138900 2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138904 2551 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138907 2551 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138911 2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138914 2551 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138917 2551 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138921 2551 flags.go:64] FLAG: --enable-load-reader="false" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138924 2551 flags.go:64] FLAG: --enable-server="true" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138929 2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138933 2551 flags.go:64] FLAG: --event-burst="100" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138936 2551 flags.go:64] FLAG: --event-qps="50" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138939 2551 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138943 2551 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138946 2551 flags.go:64] FLAG: --eviction-hard="" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138949 2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138952 2551 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138955 2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138958 2551 flags.go:64] FLAG: --eviction-soft="" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138961 2551 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138964 2551 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138967 2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138970 2551 flags.go:64] FLAG: --experimental-mounter-path="" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138973 2551 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138976 2551 flags.go:64] FLAG: --fail-swap-on="true" Jul 17 10:52:15.472256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138979 2551 flags.go:64] FLAG: --feature-gates="" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138983 2551 flags.go:64] FLAG: --file-check-frequency="20s" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138986 2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138990 2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138993 2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138996 2551 flags.go:64] FLAG: --healthz-port="10248" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.138999 2551 flags.go:64] FLAG: --help="false" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139001 2551 flags.go:64] FLAG: --hostname-override="ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139005 2551 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139007 2551 flags.go:64] FLAG: --http-check-frequency="20s" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139010 2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139013 2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139017 2551 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139020 2551 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139024 2551 flags.go:64] FLAG: --image-service-endpoint="" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139027 2551 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139031 2551 flags.go:64] FLAG: --kube-api-burst="100" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139034 2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139037 2551 flags.go:64] FLAG: --kube-api-qps="50" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139040 2551 flags.go:64] FLAG: --kube-reserved="" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139043 2551 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139046 2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139048 2551 flags.go:64] FLAG: --kubelet-cgroups="" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139051 2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 17 10:52:15.473365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139054 2551 flags.go:64] FLAG: --lock-file="" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139057 2551 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139060 2551 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139064 2551 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139069 2551 flags.go:64] FLAG: --log-json-split-stream="false" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139073 2551 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139076 2551 flags.go:64] FLAG: --log-text-split-stream="false" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139079 2551 flags.go:64] FLAG: --logging-format="text" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139082 2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139086 2551 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139088 2551 flags.go:64] FLAG: --manifest-url="" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139091 2551 flags.go:64] FLAG: --manifest-url-header="" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139095 2551 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139098 2551 flags.go:64] FLAG: --max-open-files="1000000" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139102 2551 flags.go:64] FLAG: --max-pods="110" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139105 2551 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139108 2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139111 2551 flags.go:64] FLAG: --memory-manager-policy="None" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139113 2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139116 2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139119 2551 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139122 2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139131 2551 flags.go:64] FLAG: --node-status-max-images="50" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139134 2551 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139138 2551 flags.go:64] FLAG: --oom-score-adj="-999" Jul 17 10:52:15.474475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139141 2551 flags.go:64] FLAG: --pod-cidr="" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139144 2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139150 2551 flags.go:64] FLAG: --pod-manifest-path="" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139153 2551 flags.go:64] FLAG: --pod-max-pids="-1" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139156 2551 flags.go:64] FLAG: --pods-per-core="0" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139158 2551 flags.go:64] FLAG: --port="10250" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139161 2551 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139164 2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-098df612de66de27f" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139168 2551 flags.go:64] FLAG: --qos-reserved="" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139170 2551 flags.go:64] FLAG: --read-only-port="10255" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139173 2551 flags.go:64] FLAG: --register-node="true" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139176 2551 flags.go:64] FLAG: --register-with-taints="" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139180 2551 flags.go:64] FLAG: --registry-burst="10" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139190 2551 flags.go:64] FLAG: --registry-qps="5" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139193 2551 flags.go:64] FLAG: --reserved-cpus="" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139196 2551 flags.go:64] FLAG: --reserved-memory="" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139199 2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139202 2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139205 2551 flags.go:64] FLAG: --rotate-certificates="false" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139208 2551 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139211 2551 flags.go:64] FLAG: --runonce="false" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139214 2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139216 2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139220 2551 flags.go:64] FLAG: --seccomp-default="false" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139223 2551 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 17 10:52:15.478138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139225 2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139228 2551 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139231 2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139235 2551 flags.go:64] FLAG: --storage-driver-password="root" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139237 2551 flags.go:64] FLAG: --storage-driver-secure="false" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139241 2551 flags.go:64] FLAG: --storage-driver-table="stats" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139246 2551 flags.go:64] FLAG: --storage-driver-user="root" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139248 2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139251 2551 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139254 2551 flags.go:64] FLAG: --system-cgroups="" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139257 2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139262 2551 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139265 2551 flags.go:64] FLAG: --tls-cert-file="" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139267 2551 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139272 2551 flags.go:64] FLAG: --tls-min-version="" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139275 2551 flags.go:64] FLAG: --tls-private-key-file="" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139278 2551 flags.go:64] FLAG: --topology-manager-policy="none" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139281 2551 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139283 2551 flags.go:64] FLAG: --topology-manager-scope="container" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139286 2551 flags.go:64] FLAG: --v="2" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139290 2551 flags.go:64] FLAG: --version="false" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139296 2551 flags.go:64] FLAG: --vmodule="" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139299 2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.139302 2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139418 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 10:52:15.479249 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139422 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139425 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139428 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139431 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139434 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139436 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139439 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139442 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139444 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139447 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139449 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139452 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139456 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139460 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139462 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139465 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139468 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139470 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139473 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139475 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 10:52:15.486363 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139478 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139480 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139483 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139487 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139490 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139506 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139509 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139512 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139515 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139518 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139521 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139523 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139526 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139529 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139531 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139534 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139536 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139539 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139541 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 10:52:15.487251 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139544 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139546 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139549 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139552 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139555 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139558 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139562 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139565 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139568 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139570 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139573 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139575 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139578 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139580 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139583 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139585 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139589 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139592 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139595 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139597 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 10:52:15.488125 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139600 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139603 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139606 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139609 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139611 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139614 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139624 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139628 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139631 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139633 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139636 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139638 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139641 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139644 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139646 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139649 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139651 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139655 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139660 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139662 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 10:52:15.488999 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139665 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139667 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139670 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139672 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139675 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139678 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139680 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139682 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139685 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139688 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139692 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139695 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139699 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139702 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139705 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139708 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139711 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139714 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 10:52:15.489872 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.139717 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.140424 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.147800 2551 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.147820 2551 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.147839 2551 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.147846 2551 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147915 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147922 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147926 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147929 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147932 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147935 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147938 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147941 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 10:52:15.490740 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147944 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147947 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147950 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147954 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147957 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147960 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147962 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147965 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147967 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147970 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147972 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147975 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147978 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147981 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147984 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147987 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147989 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147992 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147994 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147997 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 10:52:15.491404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.147999 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148002 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148005 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148007 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148009 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148012 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148015 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148017 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148020 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148022 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148024 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148027 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148030 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148033 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148036 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148038 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148041 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148043 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148048 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148051 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 10:52:15.492277 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148054 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148056 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148058 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148061 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148064 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148066 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148069 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148071 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148073 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148076 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148078 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148080 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148083 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148086 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148089 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148091 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148094 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148096 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148098 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148101 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 10:52:15.493167 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148104 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148106 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148110 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148113 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148117 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148120 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148122 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148125 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148128 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148131 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148136 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148140 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148142 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148145 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148148 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148150 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148153 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148155 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148157 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 10:52:15.494265 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148160 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148162 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148165 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148167 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148170 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148172 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148175 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148177 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148180 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148183 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148185 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148188 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.148193 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148308 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148312 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 10:52:15.495138 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148315 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148318 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148321 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148324 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148328 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148331 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148334 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148337 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148341 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148344 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148347 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148349 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148352 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148354 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148357 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148359 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148362 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148364 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148367 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148369 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 10:52:15.496027 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148372 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148374 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148376 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148379 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148381 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148385 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148389 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148393 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148398 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148400 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148403 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148406 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148409 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148412 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148414 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148417 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148420 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148423 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148425 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148427 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 10:52:15.496910 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148434 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148437 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148440 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148442 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148445 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148447 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148450 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148453 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148455 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148457 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148460 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148462 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148465 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148467 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148470 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148472 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148475 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148477 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148480 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 10:52:15.497805 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148483 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148487 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148490 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148492 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148511 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148514 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148516 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148519 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148522 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148524 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148526 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148529 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148532 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148536 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148539 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148542 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148544 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148547 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148549 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148552 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 10:52:15.498688 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148554 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148557 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148559 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148562 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148565 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148567 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148569 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148572 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148576 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148579 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148582 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148585 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148588 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148592 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148595 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148598 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148601 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 10:52:15.499771 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:15.148603 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.148608 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.148749 2551 server.go:962] "Client rotation is on, will bootstrap in background" Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.151102 2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.152837 2551 server.go:1019] "Starting client certificate rotation" Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.152952 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.152989 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.155891 2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.158061 2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.170558 2551 log.go:25] "Validated CRI v1 runtime API" Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.173258 2551 log.go:25] "Validated CRI v1 image API" Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.174130 2551 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 17 10:52:15.500435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.179871 2551 fs.go:135] Filesystem UUIDs: map[705bf7d2-36b9-4905-9f34-760e45cb19d1:/dev/nvme0n1p3 7577dfd2-14c9-4245-bebb-f22d7f248089:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 17 10:52:15.501096 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.179887 2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 17 10:52:15.610934 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.185767 2551 manager.go:216] Machine: {Timestamp:2026-07-17 10:52:15.184332649 +0000 UTC m=+0.382899047 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3111938 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f63f900e2242455351a1657ff2d0d SystemUUID:ec2f63f9-00e2-2424-5535-1a1657ff2d0d BootID:9e9f6b3c-0487-4d4e-8f00-b04a73fdfcfc Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f9:c6:9f:38:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f9:c6:9f:38:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:9a:b6:83:3b:6f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 17 10:52:15.610934 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.185873 2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 17 10:52:15.610934 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.185904 2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 10:52:15.610934 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.185954 2551 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 17 10:52:15.610934 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.189676 2551 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 17 10:52:15.610934 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.189701 2551 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-42.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 17 10:52:15.610934 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.190533 2551 topology_manager.go:138] "Creating topology manager with none policy" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.190541 2551 container_manager_linux.go:309] "Creating device plugin manager" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.190549 2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.190564 2551 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.192455 2551 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.199478 2551 state_mem.go:36] "Initialized new in-memory state store" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.199622 2551 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.199649 2551 kubelet.go:486] "Attempting to sync node with API server" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.199660 2551 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.199676 2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.199687 2551 kubelet.go:398] "Adding apiserver pod source" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.199696 2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.200074 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.200088 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.202511 2551 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.205033 2551 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.205060 2551 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206515 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206529 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206535 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206541 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206553 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206559 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206580 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206593 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206600 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206605 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206623 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.206637 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.208411 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.208419 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.209460 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.209460 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.211248 2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-42.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.211764 2551 server.go:1268] "Started kubelet" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.212131 2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.214143 2551 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.214310 2551 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.215298 2551 server.go:310] "Adding debug handlers to kubelet server" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.215887 2551 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 17 10:52:15.611890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.221576 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q6lhw" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.221933 2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.225643 2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.225666 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.225731 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226367 2551 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226365 2551 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226383 2551 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226423 2551 factory.go:55] Registering systemd factory Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226439 2551 factory.go:223] Registration of the systemd container factory successfully Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226468 2551 reconstruct.go:97] "Volume reconstruction finished" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226477 2551 reconciler.go:29] "Reconciler: start to sync state" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226623 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q6lhw" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226658 2551 factory.go:153] Registering CRI-O factory Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226673 2551 factory.go:223] Registration of the crio container factory successfully Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226724 2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226747 2551 factory.go:103] Registering Raw factory Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.226762 2551 manager.go:1196] Started watching for new ooms in manager Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.227263 2551 manager.go:349] Starting recovery of all containers Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.232815 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.232926 2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.232809 2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-42.ec2.internal.18c30e3da029cd51 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-42.ec2.internal,UID:ip-10-0-143-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-42.ec2.internal,},FirstTimestamp:2026-07-17 10:52:15.211736401 +0000 UTC m=+0.410302800,LastTimestamp:2026-07-17 10:52:15.211736401 +0000 UTC m=+0.410302800,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-42.ec2.internal,}" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.245001 2551 manager.go:354] Recovery completed Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.249920 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.251821 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.251853 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.251869 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.252367 2551 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.252376 2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.252392 2551 state_mem.go:36] "Initialized new in-memory state store" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.252725 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.253580 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 17 10:52:15.617274 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.253609 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.253623 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.254533 2551 policy_none.go:49] "None policy: Start" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.254549 2551 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.254562 2551 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.256646 2551 policy_none.go:47] "Start" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.289680 2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.291350 2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.291398 2551 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.291420 2551 kubelet.go:2448] "Starting kubelet main sync loop" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.291460 2551 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.293735 2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.307082 2551 manager.go:337] "Starting Device Plugin manager" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.307113 2551 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.307122 2551 server.go:87] "Starting device plugin registration server" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.307404 2551 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.307414 2551 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.307523 2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.307598 2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.307607 2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.308023 2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.308061 2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.392309 2551 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal"] Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.392415 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.393477 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.393521 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.393537 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.395837 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.395961 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.396003 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.396593 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.396622 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.396635 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.396592 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.396678 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.396690 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.398886 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.398908 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 10:52:15.618817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.400432 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.400457 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.400471 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.407521 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.408190 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientMemory" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.408214 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.408228 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeHasSufficientPID" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.408248 2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.416257 2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.416274 2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-42.ec2.internal\": node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.427821 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5927dc7dc1f46c8653401b3916c25227-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"5927dc7dc1f46c8653401b3916c25227\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.427844 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5927dc7dc1f46c8653401b3916c25227-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"5927dc7dc1f46c8653401b3916c25227\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.427861 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/683047144508312a081996d7cfb41351-config\") pod \"kube-apiserver-proxy-ip-10-0-143-42.ec2.internal\" (UID: \"683047144508312a081996d7cfb41351\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.441046 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.444625 2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-42.ec2.internal\" not found" node="ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.449070 2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-42.ec2.internal\" not found" node="ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.536780 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5927dc7dc1f46c8653401b3916c25227-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"5927dc7dc1f46c8653401b3916c25227\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.536824 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5927dc7dc1f46c8653401b3916c25227-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"5927dc7dc1f46c8653401b3916c25227\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.536842 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/683047144508312a081996d7cfb41351-config\") pod \"kube-apiserver-proxy-ip-10-0-143-42.ec2.internal\" (UID: \"683047144508312a081996d7cfb41351\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.536884 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/683047144508312a081996d7cfb41351-config\") pod \"kube-apiserver-proxy-ip-10-0-143-42.ec2.internal\" (UID: \"683047144508312a081996d7cfb41351\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.626353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.536891 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5927dc7dc1f46c8653401b3916c25227-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"5927dc7dc1f46c8653401b3916c25227\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.627468 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.536891 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5927dc7dc1f46c8653401b3916c25227-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal\" (UID: \"5927dc7dc1f46c8653401b3916c25227\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 17 10:52:15.627468 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.541593 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:15.734251 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.641997 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:17.391027 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.742515 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:17.391027 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.748672 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 17 10:52:17.391027 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:15.760047 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 17 10:52:17.391027 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:15.843570 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:17.555488 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:17.469094 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:17.660212 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:17.569987 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:17.671584 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:17.670617 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:17.771259 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:17.676751 2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 17 10:52:17.771259 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:17.676881 2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 10:52:17.771259 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:17.758888 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 10:47:16 +0000 UTC" deadline="2028-02-16 20:01:25.429357144 +0000 UTC" Jul 17 10:52:17.771259 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:17.758913 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13905h9m7.670447164s" Jul 17 10:52:17.771259 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:17.770845 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:17.888610 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:17.870888 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:17.888610 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:17.887627 2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 10:52:17.990022 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:17.971869 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:18.106890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.041166 2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 10:52:18.106890 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.072838 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-42.ec2.internal\" not found" Jul 17 10:52:18.106890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.078421 2551 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 10:52:18.106890 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:18.080761 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5927dc7dc1f46c8653401b3916c25227.slice/crio-49e3e4054462b01069291406c5f1a745970668a0be482760f82a4c2fc22a6ac2 WatchSource:0}: Error finding container 49e3e4054462b01069291406c5f1a745970668a0be482760f82a4c2fc22a6ac2: Status 404 returned error can't find the container with id 49e3e4054462b01069291406c5f1a745970668a0be482760f82a4c2fc22a6ac2 Jul 17 10:52:18.106890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.084731 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 10:52:18.267193 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.151775 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" Jul 17 10:52:18.267193 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.163177 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 10:52:18.267193 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.164258 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" Jul 17 10:52:18.267193 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.177024 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 10:52:18.398840 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:18.361676 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683047144508312a081996d7cfb41351.slice/crio-eff47ce98f47924dc87dceeabf674c34edfed937432505653eada9b60c43df78 WatchSource:0}: Error finding container eff47ce98f47924dc87dceeabf674c34edfed937432505653eada9b60c43df78: Status 404 returned error can't find the container with id eff47ce98f47924dc87dceeabf674c34edfed937432505653eada9b60c43df78 Jul 17 10:52:18.726754 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.726710 2551 apiserver.go:52] "Watching apiserver" Jul 17 10:52:18.735904 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.735868 2551 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 17 10:52:18.736826 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.736791 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal","openshift-multus/multus-additional-cni-plugins-4j4j7","openshift-multus/multus-ms4m6","openshift-network-diagnostics/network-check-target-9p5ns","kube-system/global-pull-secret-syncer-6m5q9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5","openshift-image-registry/node-ca-8qjkv","openshift-multus/network-metrics-daemon-sqv6x","openshift-network-operator/iptables-alerter-mzjpx","openshift-ovn-kubernetes/ovnkube-node-vxr2t","kube-system/konnectivity-agent-ffbvd","kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal","openshift-cluster-node-tuning-operator/tuned-lslhc","openshift-dns/node-resolver-bsl8q"] Jul 17 10:52:18.739461 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.739438 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.742269 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.742245 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 17 10:52:18.742390 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.742275 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 17 10:52:18.742390 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.742304 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pd87q\"" Jul 17 10:52:18.742390 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.742245 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 17 10:52:18.742587 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.742571 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 17 10:52:18.742641 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.742579 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.743178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.743159 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 17 10:52:18.743308 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.743194 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 17 10:52:18.744741 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.744724 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 17 10:52:18.744989 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.744965 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 17 10:52:18.745164 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.745146 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.745922 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.745382 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d4spl\"" Jul 17 10:52:18.745922 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.745776 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 17 10:52:18.747142 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.747124 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 17 10:52:18.747142 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.747140 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8qjkv" Jul 17 10:52:18.747597 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.747491 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9fdpn\"" Jul 17 10:52:18.747954 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.747602 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 17 10:52:18.747954 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.747778 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 17 10:52:18.747954 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.747787 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 17 10:52:18.749848 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.749413 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.749848 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.749610 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 17 10:52:18.749848 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.749612 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 17 10:52:18.749848 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.749754 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 17 10:52:18.749848 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.749835 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-br8zw\"" Jul 17 10:52:18.751039 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.751019 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 17 10:52:18.751681 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.751661 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 17 10:52:18.751778 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.751736 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 17 10:52:18.751833 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.751739 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-v2j8r\"" Jul 17 10:52:18.752137 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.752114 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:18.752217 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.752193 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:18.754600 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.754523 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mzjpx" Jul 17 10:52:18.757078 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.757061 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:18.757162 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.757139 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:18.758365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.758344 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9cgqn\"" Jul 17 10:52:18.758582 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.758563 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 17 10:52:18.758679 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.758574 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 17 10:52:18.759173 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.759159 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 17 10:52:18.759845 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.759830 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.761852 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.761809 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:18.762177 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.762071 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 17 10:52:18.762177 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.762093 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 17 10:52:18.762310 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.762294 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2szhr\"" Jul 17 10:52:18.764273 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.764253 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 17 10:52:18.764411 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.764392 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9x5n6\"" Jul 17 10:52:18.765502 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.765098 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 17 10:52:18.765502 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.765322 2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 10:52:18.767505 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.767479 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:18.767896 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.767675 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:18.767896 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.767790 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bsl8q" Jul 17 10:52:18.770674 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.770650 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 17 10:52:18.770989 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.770881 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lvb6d\"" Jul 17 10:52:18.770989 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.770900 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 17 10:52:18.774685 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774664 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-var-lib-openvswitch\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.774765 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774694 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-run-ovn-kubernetes\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.774765 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774712 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-run\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.774765 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774734 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-lib-modules\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.774917 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774779 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmr28\" (UniqueName: \"kubernetes.io/projected/8cceb304-a674-42a1-9620-3d940dc45f70-kube-api-access-kmr28\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.774917 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774810 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/91a1ec59-ea70-461a-a9be-471315c4c7df-kubelet-config\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:18.774917 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774836 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:18.774917 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774857 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-run-systemd\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.774917 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774876 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.774917 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774898 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88f00775-a242-4da4-bfc9-c73a05c78e24-host\") pod \"node-ca-8qjkv\" (UID: \"88f00775-a242-4da4-bfc9-c73a05c78e24\") " pod="openshift-image-registry/node-ca-8qjkv" Jul 17 10:52:18.775178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774921 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-kubernetes\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.775178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774938 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-kubelet\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.775178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774952 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-socket-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.775178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774969 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-var-lib-cni-bin\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.775178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.774989 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-sysctl-d\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.775178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775034 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-cni-bin\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.775178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775068 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.775178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775094 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-registration-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.775178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775121 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.775178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775143 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-cni-dir\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.775178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775166 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-socket-dir-parent\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775187 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-var-lib-kubelet\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775220 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-os-release\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775243 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-systemd\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775265 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-sys\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775289 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45lnj\" (UniqueName: \"kubernetes.io/projected/bbc3b057-6133-4198-9876-fd12469991a2-kube-api-access-45lnj\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775313 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88f00775-a242-4da4-bfc9-c73a05c78e24-serviceca\") pod \"node-ca-8qjkv\" (UID: \"88f00775-a242-4da4-bfc9-c73a05c78e24\") " pod="openshift-image-registry/node-ca-8qjkv" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775336 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-slash\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775360 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bbb466e6-585e-4eb8-8409-9839b0a98861-ovnkube-config\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775383 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbb466e6-585e-4eb8-8409-9839b0a98861-ovn-node-metrics-cert\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775406 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvzm6\" (UniqueName: \"kubernetes.io/projected/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-kube-api-access-kvzm6\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775431 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-run-netns\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775469 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-daemon-config\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775498 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-node-log\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.775689 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775523 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bbb466e6-585e-4eb8-8409-9839b0a98861-env-overrides\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775564 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q29kz\" (UniqueName: \"kubernetes.io/projected/bbb466e6-585e-4eb8-8409-9839b0a98861-kube-api-access-q29kz\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775609 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-cni-binary-copy\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775642 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-run-k8s-cni-cncf-io\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775668 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-var-lib-kubelet\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775690 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/91a1ec59-ea70-461a-a9be-471315c4c7df-dbus\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775704 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-run-openvswitch\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775719 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cceb304-a674-42a1-9620-3d940dc45f70-tmp\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775739 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-host\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775759 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-systemd-units\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775776 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-etc-kubernetes\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775792 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-device-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775809 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-system-cni-dir\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775829 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-run-multus-certs\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.776266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775850 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-log-socket\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775870 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-sys-fs\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775912 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-cnibin\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775939 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-var-lib-cni-multus\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.775972 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bbb466e6-585e-4eb8-8409-9839b0a98861-ovnkube-script-lib\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.776005 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-hostroot\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.776047 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-sysctl-conf\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.776073 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-run-netns\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.776094 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-cni-netd\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.776117 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j55gs\" (UniqueName: \"kubernetes.io/projected/88f00775-a242-4da4-bfc9-c73a05c78e24-kube-api-access-j55gs\") pod \"node-ca-8qjkv\" (UID: \"88f00775-a242-4da4-bfc9-c73a05c78e24\") " pod="openshift-image-registry/node-ca-8qjkv" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.776137 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-modprobe-d\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.776158 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-etc-openvswitch\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.776181 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8cceb304-a674-42a1-9620-3d940dc45f70-etc-tuned\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.776200 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-run-ovn\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.776734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.776221 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-conf-dir\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.777172 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.776244 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-sysconfig\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.786975 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.786954 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ngm5x" Jul 17 10:52:18.794134 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.793939 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ngm5x" Jul 17 10:52:18.822477 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.822432 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" event={"ID":"683047144508312a081996d7cfb41351","Type":"ContainerStarted","Data":"eff47ce98f47924dc87dceeabf674c34edfed937432505653eada9b60c43df78"} Jul 17 10:52:18.824100 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.824038 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" event={"ID":"5927dc7dc1f46c8653401b3916c25227","Type":"ContainerStarted","Data":"49e3e4054462b01069291406c5f1a745970668a0be482760f82a4c2fc22a6ac2"} Jul 17 10:52:18.852361 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.852333 2551 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 17 10:52:18.877021 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.876981 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnckj\" (UniqueName: \"kubernetes.io/projected/dff2a623-d2b3-4e48-830a-0d479108fa15-kube-api-access-rnckj\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.877021 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877022 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:18.877224 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877058 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cceb304-a674-42a1-9620-3d940dc45f70-tmp\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.877224 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877079 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-cnibin\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.877224 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877099 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-host\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.877224 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877126 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-systemd-units\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.877224 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877164 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-etc-kubernetes\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.877224 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877211 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-systemd-units\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877272 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-host\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877311 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-device-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877349 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-system-cni-dir\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877375 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-run-multus-certs\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877397 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-log-socket\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877414 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-device-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877438 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-sys-fs\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877465 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-cnibin\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877468 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-run-multus-certs\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877488 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-var-lib-cni-multus\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877510 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bbb466e6-585e-4eb8-8409-9839b0a98861-ovnkube-script-lib\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877536 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97m5l\" (UniqueName: \"kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l\") pod \"network-check-target-9p5ns\" (UID: \"5041937f-62cc-401c-9c7e-3a15f4a6e88c\") " pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877567 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-system-cni-dir\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877578 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cmm6\" (UniqueName: \"kubernetes.io/projected/5e2e9c83-f45d-44d7-b654-fd4187164f25-kube-api-access-4cmm6\") pod \"node-resolver-bsl8q\" (UID: \"5e2e9c83-f45d-44d7-b654-fd4187164f25\") " pod="openshift-dns/node-resolver-bsl8q" Jul 17 10:52:18.877748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877628 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-cnibin\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877625 2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877647 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-var-lib-cni-multus\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877659 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-hostroot\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877705 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-sysctl-conf\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877730 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-run-netns\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877753 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-cni-netd\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877781 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5e2e9c83-f45d-44d7-b654-fd4187164f25-hosts-file\") pod \"node-resolver-bsl8q\" (UID: \"5e2e9c83-f45d-44d7-b654-fd4187164f25\") " pod="openshift-dns/node-resolver-bsl8q" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877807 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j55gs\" (UniqueName: \"kubernetes.io/projected/88f00775-a242-4da4-bfc9-c73a05c78e24-kube-api-access-j55gs\") pod \"node-ca-8qjkv\" (UID: \"88f00775-a242-4da4-bfc9-c73a05c78e24\") " pod="openshift-image-registry/node-ca-8qjkv" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877831 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-modprobe-d\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877855 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-etc-openvswitch\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877881 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/339c2065-6a0f-4b69-b150-b18181ebb1f8-konnectivity-ca\") pod \"konnectivity-agent-ffbvd\" (UID: \"339c2065-6a0f-4b69-b150-b18181ebb1f8\") " pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877891 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-log-socket\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877913 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8cceb304-a674-42a1-9620-3d940dc45f70-etc-tuned\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877954 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-run-ovn\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.878447 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877961 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-sys-fs\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877986 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3c452207-798d-430e-8045-7548762dcbc2-iptables-alerter-script\") pod \"iptables-alerter-mzjpx\" (UID: \"3c452207-798d-430e-8045-7548762dcbc2\") " pod="openshift-network-operator/iptables-alerter-mzjpx" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878004 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-hostroot\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878018 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-conf-dir\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878061 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-sysconfig\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878088 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-var-lib-openvswitch\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878113 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-run-ovn-kubernetes\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878136 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-os-release\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878162 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-run\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878186 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-lib-modules\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878209 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-run-netns\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878236 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-modprobe-d\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878212 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmr28\" (UniqueName: \"kubernetes.io/projected/8cceb304-a674-42a1-9620-3d940dc45f70-kube-api-access-kmr28\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878280 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/91a1ec59-ea70-461a-a9be-471315c4c7df-kubelet-config\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:18.879117 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878312 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878343 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-run-systemd\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878307 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-cni-netd\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878374 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-466pt\" (UniqueName: \"kubernetes.io/projected/3c452207-798d-430e-8045-7548762dcbc2-kube-api-access-466pt\") pod \"iptables-alerter-mzjpx\" (UID: \"3c452207-798d-430e-8045-7548762dcbc2\") " pod="openshift-network-operator/iptables-alerter-mzjpx" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878137 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-sysctl-conf\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878461 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-etc-openvswitch\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878280 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-var-lib-openvswitch\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.877829 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-etc-kubernetes\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878605 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-run-ovn-kubernetes\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878639 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dff2a623-d2b3-4e48-830a-0d479108fa15-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878663 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-sysconfig\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878690 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878721 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-run-systemd\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878723 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88f00775-a242-4da4-bfc9-c73a05c78e24-host\") pod \"node-ca-8qjkv\" (UID: \"88f00775-a242-4da4-bfc9-c73a05c78e24\") " pod="openshift-image-registry/node-ca-8qjkv" Jul 17 10:52:18.879669 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878761 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88f00775-a242-4da4-bfc9-c73a05c78e24-host\") pod \"node-ca-8qjkv\" (UID: \"88f00775-a242-4da4-bfc9-c73a05c78e24\") " pod="openshift-image-registry/node-ca-8qjkv" Jul 17 10:52:18.880289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.878768 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-kubernetes\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.880289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.879772 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-kubelet\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.880289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.879814 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-socket-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.880289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.879847 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-var-lib-cni-bin\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.880289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.879880 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-sysctl-d\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.880289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.879919 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-cni-bin\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.880289 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.878793 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:18.880615 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.880390 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.880615 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.880426 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5227\" (UniqueName: \"kubernetes.io/projected/c9bb4909-4714-4484-96fb-e258962dfd4b-kube-api-access-c5227\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:18.880615 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.880459 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e2e9c83-f45d-44d7-b654-fd4187164f25-tmp-dir\") pod \"node-resolver-bsl8q\" (UID: \"5e2e9c83-f45d-44d7-b654-fd4187164f25\") " pod="openshift-dns/node-resolver-bsl8q" Jul 17 10:52:18.880615 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.880494 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-registration-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.880615 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.880526 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.880615 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.880590 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-run-ovn\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.880883 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.880654 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret podName:91a1ec59-ea70-461a-a9be-471315c4c7df nodeName:}" failed. No retries permitted until 2026-07-17 10:52:19.380622216 +0000 UTC m=+3.054137860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret") pod "global-pull-secret-syncer-6m5q9" (UID: "91a1ec59-ea70-461a-a9be-471315c4c7df") : object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:18.880883 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.880816 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bbb466e6-585e-4eb8-8409-9839b0a98861-ovnkube-script-lib\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.881594 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.881445 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-cni-dir\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.881594 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.881587 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-cni-bin\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.881882 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-run\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.881981 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-lib-modules\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882029 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/91a1ec59-ea70-461a-a9be-471315c4c7df-kubelet-config\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882104 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-kubernetes\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882189 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-kubelet\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882292 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-socket-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882332 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-var-lib-cni-bin\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882414 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-sysctl-d\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882479 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-conf-dir\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882538 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882682 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-cni-dir\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882756 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882816 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-registration-dir\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882819 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-socket-dir-parent\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.882978 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882933 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bbc3b057-6133-4198-9876-fd12469991a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.882922 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-var-lib-kubelet\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883018 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-socket-dir-parent\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883139 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-os-release\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883216 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8cceb304-a674-42a1-9620-3d940dc45f70-etc-tuned\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883230 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-systemd\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883286 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-sys\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883321 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-var-lib-kubelet\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883392 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45lnj\" (UniqueName: \"kubernetes.io/projected/bbc3b057-6133-4198-9876-fd12469991a2-kube-api-access-45lnj\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883415 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-etc-systemd\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883419 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88f00775-a242-4da4-bfc9-c73a05c78e24-serviceca\") pod \"node-ca-8qjkv\" (UID: \"88f00775-a242-4da4-bfc9-c73a05c78e24\") " pod="openshift-image-registry/node-ca-8qjkv" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883502 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-sys\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883542 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-slash\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.883704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883625 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bbb466e6-585e-4eb8-8409-9839b0a98861-ovnkube-config\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.884281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883725 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbb466e6-585e-4eb8-8409-9839b0a98861-ovn-node-metrics-cert\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.884281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883802 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-system-cni-dir\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.884281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883836 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dff2a623-d2b3-4e48-830a-0d479108fa15-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.884281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883883 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvzm6\" (UniqueName: \"kubernetes.io/projected/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-kube-api-access-kvzm6\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.884281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.883990 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-host-slash\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.884281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884050 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-run-netns\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.884281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884085 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-daemon-config\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.884281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884118 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-node-log\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.884281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884149 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bbb466e6-585e-4eb8-8409-9839b0a98861-env-overrides\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.884281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884179 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q29kz\" (UniqueName: \"kubernetes.io/projected/bbb466e6-585e-4eb8-8409-9839b0a98861-kube-api-access-q29kz\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.884281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884211 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.884812 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884313 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/339c2065-6a0f-4b69-b150-b18181ebb1f8-agent-certs\") pod \"konnectivity-agent-ffbvd\" (UID: \"339c2065-6a0f-4b69-b150-b18181ebb1f8\") " pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:18.884812 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884356 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-cni-binary-copy\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.884812 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884383 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-run-k8s-cni-cncf-io\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.884812 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884406 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-os-release\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.884812 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884417 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-var-lib-kubelet\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.884812 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884466 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/91a1ec59-ea70-461a-a9be-471315c4c7df-dbus\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:18.884812 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884789 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-run-openvswitch\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.885137 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884871 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/91a1ec59-ea70-461a-a9be-471315c4c7df-dbus\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:18.885137 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.884899 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88f00775-a242-4da4-bfc9-c73a05c78e24-serviceca\") pod \"node-ca-8qjkv\" (UID: \"88f00775-a242-4da4-bfc9-c73a05c78e24\") " pod="openshift-image-registry/node-ca-8qjkv" Jul 17 10:52:18.885137 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.885123 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-run-openvswitch\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.885276 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.885164 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c452207-798d-430e-8045-7548762dcbc2-host-slash\") pod \"iptables-alerter-mzjpx\" (UID: \"3c452207-798d-430e-8045-7548762dcbc2\") " pod="openshift-network-operator/iptables-alerter-mzjpx" Jul 17 10:52:18.885276 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.885199 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dff2a623-d2b3-4e48-830a-0d479108fa15-cni-binary-copy\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.885376 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.885299 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-run-netns\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.886332 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.885533 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-host-run-k8s-cni-cncf-io\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.886332 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.885636 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bbb466e6-585e-4eb8-8409-9839b0a98861-ovnkube-config\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.886332 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.885687 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bbb466e6-585e-4eb8-8409-9839b0a98861-node-log\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.886332 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.885704 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cceb304-a674-42a1-9620-3d940dc45f70-var-lib-kubelet\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.886332 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.885764 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bbb466e6-585e-4eb8-8409-9839b0a98861-env-overrides\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.886332 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.886072 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-multus-daemon-config\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.886797 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.886365 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-cni-binary-copy\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.889994 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.889171 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cceb304-a674-42a1-9620-3d940dc45f70-tmp\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.889994 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.889473 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbb466e6-585e-4eb8-8409-9839b0a98861-ovn-node-metrics-cert\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.889994 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.889985 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmr28\" (UniqueName: \"kubernetes.io/projected/8cceb304-a674-42a1-9620-3d940dc45f70-kube-api-access-kmr28\") pod \"tuned-lslhc\" (UID: \"8cceb304-a674-42a1-9620-3d940dc45f70\") " pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:18.890198 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.890162 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j55gs\" (UniqueName: \"kubernetes.io/projected/88f00775-a242-4da4-bfc9-c73a05c78e24-kube-api-access-j55gs\") pod \"node-ca-8qjkv\" (UID: \"88f00775-a242-4da4-bfc9-c73a05c78e24\") " pod="openshift-image-registry/node-ca-8qjkv" Jul 17 10:52:18.892467 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.892426 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q29kz\" (UniqueName: \"kubernetes.io/projected/bbb466e6-585e-4eb8-8409-9839b0a98861-kube-api-access-q29kz\") pod \"ovnkube-node-vxr2t\" (UID: \"bbb466e6-585e-4eb8-8409-9839b0a98861\") " pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:18.892856 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.892682 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45lnj\" (UniqueName: \"kubernetes.io/projected/bbc3b057-6133-4198-9876-fd12469991a2-kube-api-access-45lnj\") pod \"aws-ebs-csi-driver-node-7kml5\" (UID: \"bbc3b057-6133-4198-9876-fd12469991a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:18.893609 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.893503 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvzm6\" (UniqueName: \"kubernetes.io/projected/eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd-kube-api-access-kvzm6\") pod \"multus-ms4m6\" (UID: \"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd\") " pod="openshift-multus/multus-ms4m6" Jul 17 10:52:18.953101 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.953061 2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 10:52:18.986542 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986509 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97m5l\" (UniqueName: \"kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l\") pod \"network-check-target-9p5ns\" (UID: \"5041937f-62cc-401c-9c7e-3a15f4a6e88c\") " pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:18.986777 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986564 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cmm6\" (UniqueName: \"kubernetes.io/projected/5e2e9c83-f45d-44d7-b654-fd4187164f25-kube-api-access-4cmm6\") pod \"node-resolver-bsl8q\" (UID: \"5e2e9c83-f45d-44d7-b654-fd4187164f25\") " pod="openshift-dns/node-resolver-bsl8q" Jul 17 10:52:18.986777 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986594 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5e2e9c83-f45d-44d7-b654-fd4187164f25-hosts-file\") pod \"node-resolver-bsl8q\" (UID: \"5e2e9c83-f45d-44d7-b654-fd4187164f25\") " pod="openshift-dns/node-resolver-bsl8q" Jul 17 10:52:18.986777 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986621 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/339c2065-6a0f-4b69-b150-b18181ebb1f8-konnectivity-ca\") pod \"konnectivity-agent-ffbvd\" (UID: \"339c2065-6a0f-4b69-b150-b18181ebb1f8\") " pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:18.986777 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986648 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3c452207-798d-430e-8045-7548762dcbc2-iptables-alerter-script\") pod \"iptables-alerter-mzjpx\" (UID: \"3c452207-798d-430e-8045-7548762dcbc2\") " pod="openshift-network-operator/iptables-alerter-mzjpx" Jul 17 10:52:18.986777 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986676 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-os-release\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.986777 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986718 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-466pt\" (UniqueName: \"kubernetes.io/projected/3c452207-798d-430e-8045-7548762dcbc2-kube-api-access-466pt\") pod \"iptables-alerter-mzjpx\" (UID: \"3c452207-798d-430e-8045-7548762dcbc2\") " pod="openshift-network-operator/iptables-alerter-mzjpx" Jul 17 10:52:18.986777 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986744 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dff2a623-d2b3-4e48-830a-0d479108fa15-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.987114 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986778 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5227\" (UniqueName: \"kubernetes.io/projected/c9bb4909-4714-4484-96fb-e258962dfd4b-kube-api-access-c5227\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:18.987114 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986812 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e2e9c83-f45d-44d7-b654-fd4187164f25-tmp-dir\") pod \"node-resolver-bsl8q\" (UID: \"5e2e9c83-f45d-44d7-b654-fd4187164f25\") " pod="openshift-dns/node-resolver-bsl8q" Jul 17 10:52:18.987114 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986849 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-system-cni-dir\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.987114 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986872 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dff2a623-d2b3-4e48-830a-0d479108fa15-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.987114 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986902 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.987114 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986924 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/339c2065-6a0f-4b69-b150-b18181ebb1f8-agent-certs\") pod \"konnectivity-agent-ffbvd\" (UID: \"339c2065-6a0f-4b69-b150-b18181ebb1f8\") " pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:18.987114 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986954 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c452207-798d-430e-8045-7548762dcbc2-host-slash\") pod \"iptables-alerter-mzjpx\" (UID: \"3c452207-798d-430e-8045-7548762dcbc2\") " pod="openshift-network-operator/iptables-alerter-mzjpx" Jul 17 10:52:18.987114 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.986977 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dff2a623-d2b3-4e48-830a-0d479108fa15-cni-binary-copy\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.987114 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.987010 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnckj\" (UniqueName: \"kubernetes.io/projected/dff2a623-d2b3-4e48-830a-0d479108fa15-kube-api-access-rnckj\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.987114 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.987034 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:18.987114 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.987057 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-cnibin\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.987620 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.987134 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-cnibin\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.987620 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.987512 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e2e9c83-f45d-44d7-b654-fd4187164f25-tmp-dir\") pod \"node-resolver-bsl8q\" (UID: \"5e2e9c83-f45d-44d7-b654-fd4187164f25\") " pod="openshift-dns/node-resolver-bsl8q" Jul 17 10:52:18.987620 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.987606 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-system-cni-dir\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.987772 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.987755 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/339c2065-6a0f-4b69-b150-b18181ebb1f8-konnectivity-ca\") pod \"konnectivity-agent-ffbvd\" (UID: \"339c2065-6a0f-4b69-b150-b18181ebb1f8\") " pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:18.987847 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.987831 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5e2e9c83-f45d-44d7-b654-fd4187164f25-hosts-file\") pod \"node-resolver-bsl8q\" (UID: \"5e2e9c83-f45d-44d7-b654-fd4187164f25\") " pod="openshift-dns/node-resolver-bsl8q" Jul 17 10:52:18.987905 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.987841 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3c452207-798d-430e-8045-7548762dcbc2-iptables-alerter-script\") pod \"iptables-alerter-mzjpx\" (UID: \"3c452207-798d-430e-8045-7548762dcbc2\") " pod="openshift-network-operator/iptables-alerter-mzjpx" Jul 17 10:52:18.987952 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.987902 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-os-release\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.987952 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.987904 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c452207-798d-430e-8045-7548762dcbc2-host-slash\") pod \"iptables-alerter-mzjpx\" (UID: \"3c452207-798d-430e-8045-7548762dcbc2\") " pod="openshift-network-operator/iptables-alerter-mzjpx" Jul 17 10:52:18.988053 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.988040 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dff2a623-d2b3-4e48-830a-0d479108fa15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.988292 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.988272 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dff2a623-d2b3-4e48-830a-0d479108fa15-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.988396 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.988324 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:18.988499 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.988483 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs podName:c9bb4909-4714-4484-96fb-e258962dfd4b nodeName:}" failed. No retries permitted until 2026-07-17 10:52:19.488459544 +0000 UTC m=+3.161975191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs") pod "network-metrics-daemon-sqv6x" (UID: "c9bb4909-4714-4484-96fb-e258962dfd4b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:18.988499 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.988481 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dff2a623-d2b3-4e48-830a-0d479108fa15-cni-binary-copy\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.988944 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.988903 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dff2a623-d2b3-4e48-830a-0d479108fa15-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.991153 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.990819 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/339c2065-6a0f-4b69-b150-b18181ebb1f8-agent-certs\") pod \"konnectivity-agent-ffbvd\" (UID: \"339c2065-6a0f-4b69-b150-b18181ebb1f8\") " pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:18.995171 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.995152 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 10:52:18.995273 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.995174 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 10:52:18.995273 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.995188 2551 projected.go:196] Error preparing data for projected volume kube-api-access-97m5l for pod openshift-network-diagnostics/network-check-target-9p5ns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:18.995273 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:18.995254 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l podName:5041937f-62cc-401c-9c7e-3a15f4a6e88c nodeName:}" failed. No retries permitted until 2026-07-17 10:52:19.495237319 +0000 UTC m=+3.168752970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-97m5l" (UniqueName: "kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l") pod "network-check-target-9p5ns" (UID: "5041937f-62cc-401c-9c7e-3a15f4a6e88c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:18.998053 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.998029 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnckj\" (UniqueName: \"kubernetes.io/projected/dff2a623-d2b3-4e48-830a-0d479108fa15-kube-api-access-rnckj\") pod \"multus-additional-cni-plugins-4j4j7\" (UID: \"dff2a623-d2b3-4e48-830a-0d479108fa15\") " pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:18.998205 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.998168 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5227\" (UniqueName: \"kubernetes.io/projected/c9bb4909-4714-4484-96fb-e258962dfd4b-kube-api-access-c5227\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:18.999365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.999328 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cmm6\" (UniqueName: \"kubernetes.io/projected/5e2e9c83-f45d-44d7-b654-fd4187164f25-kube-api-access-4cmm6\") pod \"node-resolver-bsl8q\" (UID: \"5e2e9c83-f45d-44d7-b654-fd4187164f25\") " pod="openshift-dns/node-resolver-bsl8q" Jul 17 10:52:18.999522 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:18.999493 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-466pt\" (UniqueName: \"kubernetes.io/projected/3c452207-798d-430e-8045-7548762dcbc2-kube-api-access-466pt\") pod \"iptables-alerter-mzjpx\" (UID: \"3c452207-798d-430e-8045-7548762dcbc2\") " pod="openshift-network-operator/iptables-alerter-mzjpx" Jul 17 10:52:19.052144 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.051851 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:19.059831 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.059633 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" Jul 17 10:52:19.070286 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.070104 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ms4m6" Jul 17 10:52:19.075793 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.075632 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8qjkv" Jul 17 10:52:19.081748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.081726 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lslhc" Jul 17 10:52:19.090775 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.090741 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mzjpx" Jul 17 10:52:19.097741 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.097719 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4j4j7" Jul 17 10:52:19.104662 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.104641 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:19.111298 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.111273 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bsl8q" Jul 17 10:52:19.389968 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.389888 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:19.390093 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:19.390052 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:19.390141 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:19.390121 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret podName:91a1ec59-ea70-461a-a9be-471315c4c7df nodeName:}" failed. No retries permitted until 2026-07-17 10:52:20.390101332 +0000 UTC m=+4.063616980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret") pod "global-pull-secret-syncer-6m5q9" (UID: "91a1ec59-ea70-461a-a9be-471315c4c7df") : object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:19.491245 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.491212 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:19.491407 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:19.491343 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:19.491467 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:19.491407 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs podName:c9bb4909-4714-4484-96fb-e258962dfd4b nodeName:}" failed. No retries permitted until 2026-07-17 10:52:20.491390447 +0000 UTC m=+4.164906099 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs") pod "network-metrics-daemon-sqv6x" (UID: "c9bb4909-4714-4484-96fb-e258962dfd4b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:19.592348 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.592302 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97m5l\" (UniqueName: \"kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l\") pod \"network-check-target-9p5ns\" (UID: \"5041937f-62cc-401c-9c7e-3a15f4a6e88c\") " pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:19.592522 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:19.592504 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 10:52:19.592607 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:19.592537 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 10:52:19.592607 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:19.592568 2551 projected.go:196] Error preparing data for projected volume kube-api-access-97m5l for pod openshift-network-diagnostics/network-check-target-9p5ns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:19.592740 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:19.592632 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l podName:5041937f-62cc-401c-9c7e-3a15f4a6e88c nodeName:}" failed. No retries permitted until 2026-07-17 10:52:20.592611923 +0000 UTC m=+4.266127552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-97m5l" (UniqueName: "kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l") pod "network-check-target-9p5ns" (UID: "5041937f-62cc-401c-9c7e-3a15f4a6e88c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:19.797276 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.797233 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 10:47:18 +0000 UTC" deadline="2028-02-28 06:48:09.642977104 +0000 UTC" Jul 17 10:52:19.797276 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:19.797267 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14179h55m49.845713425s" Jul 17 10:52:20.043057 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:20.043021 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc3b057_6133_4198_9876_fd12469991a2.slice/crio-64692dc32ed40715e2715b91a23032a72f5e9b064a821169162d13b9ff806071 WatchSource:0}: Error finding container 64692dc32ed40715e2715b91a23032a72f5e9b064a821169162d13b9ff806071: Status 404 returned error can't find the container with id 64692dc32ed40715e2715b91a23032a72f5e9b064a821169162d13b9ff806071 Jul 17 10:52:20.044261 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:20.044233 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2e9c83_f45d_44d7_b654_fd4187164f25.slice/crio-44f3ca978d1efad4cbd75d4022a3099deabb4aa598a317d35db0cc1799c1a9f3 WatchSource:0}: Error finding container 44f3ca978d1efad4cbd75d4022a3099deabb4aa598a317d35db0cc1799c1a9f3: Status 404 returned error can't find the container with id 44f3ca978d1efad4cbd75d4022a3099deabb4aa598a317d35db0cc1799c1a9f3 Jul 17 10:52:20.045013 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:20.044980 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c452207_798d_430e_8045_7548762dcbc2.slice/crio-a29a90677304d1687d2363fcc9e2f3668b517a087d09a850efac2a8a3e0d67f1 WatchSource:0}: Error finding container a29a90677304d1687d2363fcc9e2f3668b517a087d09a850efac2a8a3e0d67f1: Status 404 returned error can't find the container with id a29a90677304d1687d2363fcc9e2f3668b517a087d09a850efac2a8a3e0d67f1 Jul 17 10:52:20.048447 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:20.048349 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f00775_a242_4da4_bfc9_c73a05c78e24.slice/crio-331608322f451a7febfd0a6f8f68121d26162b6544e9a29e4b33ed61db049181 WatchSource:0}: Error finding container 331608322f451a7febfd0a6f8f68121d26162b6544e9a29e4b33ed61db049181: Status 404 returned error can't find the container with id 331608322f451a7febfd0a6f8f68121d26162b6544e9a29e4b33ed61db049181 Jul 17 10:52:20.069504 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:20.069480 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbb466e6_585e_4eb8_8409_9839b0a98861.slice/crio-f6e279ab335f6e8ff133664fb61b30016a550d49c3faae2d2f5205183d7d803b WatchSource:0}: Error finding container f6e279ab335f6e8ff133664fb61b30016a550d49c3faae2d2f5205183d7d803b: Status 404 returned error can't find the container with id f6e279ab335f6e8ff133664fb61b30016a550d49c3faae2d2f5205183d7d803b Jul 17 10:52:20.069899 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:20.069874 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb6a26fe_e2ba_4d2c_b5e3_bbca1029dfbd.slice/crio-26b62a14d2133ccf5264fc1b705455f91446a6d381c29720270d91219e99dd04 WatchSource:0}: Error finding container 26b62a14d2133ccf5264fc1b705455f91446a6d381c29720270d91219e99dd04: Status 404 returned error can't find the container with id 26b62a14d2133ccf5264fc1b705455f91446a6d381c29720270d91219e99dd04 Jul 17 10:52:20.070783 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:20.070753 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cceb304_a674_42a1_9620_3d940dc45f70.slice/crio-cd3345ca57120881b417b52dfd607497d43a88d6bfa167876c238d9d46935b70 WatchSource:0}: Error finding container cd3345ca57120881b417b52dfd607497d43a88d6bfa167876c238d9d46935b70: Status 404 returned error can't find the container with id cd3345ca57120881b417b52dfd607497d43a88d6bfa167876c238d9d46935b70 Jul 17 10:52:20.071523 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:20.071400 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod339c2065_6a0f_4b69_b150_b18181ebb1f8.slice/crio-249d45696fdff518db99938f1ba5b82df385e24622db125f527e35e33c8b9e7b WatchSource:0}: Error finding container 249d45696fdff518db99938f1ba5b82df385e24622db125f527e35e33c8b9e7b: Status 404 returned error can't find the container with id 249d45696fdff518db99938f1ba5b82df385e24622db125f527e35e33c8b9e7b Jul 17 10:52:20.072196 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:20.072178 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddff2a623_d2b3_4e48_830a_0d479108fa15.slice/crio-00cc0e053aaa746e6b8b1b8c477be071f433b093185376c2fca06aa9684bb698 WatchSource:0}: Error finding container 00cc0e053aaa746e6b8b1b8c477be071f433b093185376c2fca06aa9684bb698: Status 404 returned error can't find the container with id 00cc0e053aaa746e6b8b1b8c477be071f433b093185376c2fca06aa9684bb698 Jul 17 10:52:20.399059 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.398874 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:20.399185 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:20.399023 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:20.399185 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:20.399134 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret podName:91a1ec59-ea70-461a-a9be-471315c4c7df nodeName:}" failed. No retries permitted until 2026-07-17 10:52:22.399110966 +0000 UTC m=+6.072626609 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret") pod "global-pull-secret-syncer-6m5q9" (UID: "91a1ec59-ea70-461a-a9be-471315c4c7df") : object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:20.500228 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.499636 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:20.500228 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:20.499806 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:20.500228 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:20.499884 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs podName:c9bb4909-4714-4484-96fb-e258962dfd4b nodeName:}" failed. No retries permitted until 2026-07-17 10:52:22.499854035 +0000 UTC m=+6.173369680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs") pod "network-metrics-daemon-sqv6x" (UID: "c9bb4909-4714-4484-96fb-e258962dfd4b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:20.600165 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.600040 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97m5l\" (UniqueName: \"kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l\") pod \"network-check-target-9p5ns\" (UID: \"5041937f-62cc-401c-9c7e-3a15f4a6e88c\") " pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:20.600344 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:20.600207 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 10:52:20.600344 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:20.600227 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 10:52:20.600344 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:20.600239 2551 projected.go:196] Error preparing data for projected volume kube-api-access-97m5l for pod openshift-network-diagnostics/network-check-target-9p5ns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:20.600344 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:20.600312 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l podName:5041937f-62cc-401c-9c7e-3a15f4a6e88c nodeName:}" failed. No retries permitted until 2026-07-17 10:52:22.600291965 +0000 UTC m=+6.273807600 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-97m5l" (UniqueName: "kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l") pod "network-check-target-9p5ns" (UID: "5041937f-62cc-401c-9c7e-3a15f4a6e88c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:20.797504 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.797434 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 10:47:18 +0000 UTC" deadline="2028-03-19 22:39:10.498488005 +0000 UTC" Jul 17 10:52:20.797504 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.797504 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14675h46m49.700989939s" Jul 17 10:52:20.819640 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.819604 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:20.819789 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:20.819752 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:20.820267 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.820243 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:20.820360 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:20.820335 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:20.820420 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.820402 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:20.820488 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:20.820472 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:20.832506 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.832475 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bsl8q" event={"ID":"5e2e9c83-f45d-44d7-b654-fd4187164f25","Type":"ContainerStarted","Data":"44f3ca978d1efad4cbd75d4022a3099deabb4aa598a317d35db0cc1799c1a9f3"} Jul 17 10:52:20.843113 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.843029 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" event={"ID":"bbc3b057-6133-4198-9876-fd12469991a2","Type":"ContainerStarted","Data":"64692dc32ed40715e2715b91a23032a72f5e9b064a821169162d13b9ff806071"} Jul 17 10:52:20.848252 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.847367 2551 generic.go:356] "Generic (PLEG): container finished" podID="5927dc7dc1f46c8653401b3916c25227" containerID="cd653046c6ab2f8a5f59baffa324c733d57924520e31fd845c46145fd9844099" exitCode=0 Jul 17 10:52:20.848252 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.847439 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" event={"ID":"5927dc7dc1f46c8653401b3916c25227","Type":"ContainerDied","Data":"cd653046c6ab2f8a5f59baffa324c733d57924520e31fd845c46145fd9844099"} Jul 17 10:52:20.850786 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.850681 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4j4j7" event={"ID":"dff2a623-d2b3-4e48-830a-0d479108fa15","Type":"ContainerStarted","Data":"00cc0e053aaa746e6b8b1b8c477be071f433b093185376c2fca06aa9684bb698"} Jul 17 10:52:20.856440 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.856384 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" event={"ID":"bbb466e6-585e-4eb8-8409-9839b0a98861","Type":"ContainerStarted","Data":"f6e279ab335f6e8ff133664fb61b30016a550d49c3faae2d2f5205183d7d803b"} Jul 17 10:52:20.859632 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.859542 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mzjpx" event={"ID":"3c452207-798d-430e-8045-7548762dcbc2","Type":"ContainerStarted","Data":"a29a90677304d1687d2363fcc9e2f3668b517a087d09a850efac2a8a3e0d67f1"} Jul 17 10:52:20.864607 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.864443 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" event={"ID":"683047144508312a081996d7cfb41351","Type":"ContainerStarted","Data":"6acb6b7048ac05fa3f92874f3025d51e18be6ab4de756fe24b3743f98834f60a"} Jul 17 10:52:20.873539 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.873513 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffbvd" event={"ID":"339c2065-6a0f-4b69-b150-b18181ebb1f8","Type":"ContainerStarted","Data":"249d45696fdff518db99938f1ba5b82df385e24622db125f527e35e33c8b9e7b"} Jul 17 10:52:20.887133 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.887085 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lslhc" event={"ID":"8cceb304-a674-42a1-9620-3d940dc45f70","Type":"ContainerStarted","Data":"cd3345ca57120881b417b52dfd607497d43a88d6bfa167876c238d9d46935b70"} Jul 17 10:52:20.891142 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.891116 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ms4m6" event={"ID":"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd","Type":"ContainerStarted","Data":"26b62a14d2133ccf5264fc1b705455f91446a6d381c29720270d91219e99dd04"} Jul 17 10:52:20.903266 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:20.903236 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8qjkv" event={"ID":"88f00775-a242-4da4-bfc9-c73a05c78e24","Type":"ContainerStarted","Data":"331608322f451a7febfd0a6f8f68121d26162b6544e9a29e4b33ed61db049181"} Jul 17 10:52:21.915693 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:21.915612 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" event={"ID":"5927dc7dc1f46c8653401b3916c25227","Type":"ContainerStarted","Data":"5652da1a92e95b7d9f92b1f6c6abcce19570af3270dbf057c29c52ba2083ba11"} Jul 17 10:52:21.931217 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:21.931161 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-42.ec2.internal" podStartSLOduration=3.931139134 podStartE2EDuration="3.931139134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:52:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:20.880216694 +0000 UTC m=+4.553732347" watchObservedRunningTime="2026-07-17 10:52:21.931139134 +0000 UTC m=+5.604654788" Jul 17 10:52:22.417835 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:22.417250 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:22.417835 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:22.417396 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:22.417835 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:22.417463 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret podName:91a1ec59-ea70-461a-a9be-471315c4c7df nodeName:}" failed. No retries permitted until 2026-07-17 10:52:26.4174439 +0000 UTC m=+10.090959549 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret") pod "global-pull-secret-syncer-6m5q9" (UID: "91a1ec59-ea70-461a-a9be-471315c4c7df") : object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:22.518501 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:22.518448 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:22.518694 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:22.518677 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:22.518791 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:22.518754 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs podName:c9bb4909-4714-4484-96fb-e258962dfd4b nodeName:}" failed. No retries permitted until 2026-07-17 10:52:26.518731682 +0000 UTC m=+10.192247335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs") pod "network-metrics-daemon-sqv6x" (UID: "c9bb4909-4714-4484-96fb-e258962dfd4b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:22.619925 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:22.619883 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97m5l\" (UniqueName: \"kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l\") pod \"network-check-target-9p5ns\" (UID: \"5041937f-62cc-401c-9c7e-3a15f4a6e88c\") " pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:22.620151 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:22.620128 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 10:52:22.620251 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:22.620163 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 10:52:22.620251 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:22.620177 2551 projected.go:196] Error preparing data for projected volume kube-api-access-97m5l for pod openshift-network-diagnostics/network-check-target-9p5ns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:22.620251 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:22.620237 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l podName:5041937f-62cc-401c-9c7e-3a15f4a6e88c nodeName:}" failed. No retries permitted until 2026-07-17 10:52:26.620217638 +0000 UTC m=+10.293733271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-97m5l" (UniqueName: "kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l") pod "network-check-target-9p5ns" (UID: "5041937f-62cc-401c-9c7e-3a15f4a6e88c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:22.820121 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:22.819327 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:22.820121 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:22.819470 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:22.820121 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:22.819847 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:22.820121 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:22.819939 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:22.820121 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:22.819983 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:22.820121 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:22.820045 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:24.818190 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:24.817803 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:24.818190 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:24.817947 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:24.818190 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:24.818008 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:24.818190 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:24.818117 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:24.819057 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:24.818912 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:24.819057 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:24.819015 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:26.452630 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:26.452587 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:26.453087 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:26.452746 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:26.453087 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:26.452819 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret podName:91a1ec59-ea70-461a-a9be-471315c4c7df nodeName:}" failed. No retries permitted until 2026-07-17 10:52:34.452799545 +0000 UTC m=+18.126315189 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret") pod "global-pull-secret-syncer-6m5q9" (UID: "91a1ec59-ea70-461a-a9be-471315c4c7df") : object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:26.553831 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:26.553794 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:26.554015 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:26.553932 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:26.554015 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:26.554003 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs podName:c9bb4909-4714-4484-96fb-e258962dfd4b nodeName:}" failed. No retries permitted until 2026-07-17 10:52:34.553982619 +0000 UTC m=+18.227498268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs") pod "network-metrics-daemon-sqv6x" (UID: "c9bb4909-4714-4484-96fb-e258962dfd4b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:26.654971 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:26.654935 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97m5l\" (UniqueName: \"kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l\") pod \"network-check-target-9p5ns\" (UID: \"5041937f-62cc-401c-9c7e-3a15f4a6e88c\") " pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:26.655185 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:26.655136 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 10:52:26.655185 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:26.655160 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 10:52:26.655185 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:26.655175 2551 projected.go:196] Error preparing data for projected volume kube-api-access-97m5l for pod openshift-network-diagnostics/network-check-target-9p5ns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:26.655352 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:26.655249 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l podName:5041937f-62cc-401c-9c7e-3a15f4a6e88c nodeName:}" failed. No retries permitted until 2026-07-17 10:52:34.655230536 +0000 UTC m=+18.328746180 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-97m5l" (UniqueName: "kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l") pod "network-check-target-9p5ns" (UID: "5041937f-62cc-401c-9c7e-3a15f4a6e88c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:26.817134 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:26.817085 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:26.817353 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:26.817244 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:26.818988 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:26.817656 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:26.818988 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:26.817759 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:26.818988 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:26.818868 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:26.818988 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:26.818946 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:28.816855 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:28.816821 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:28.817399 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:28.816832 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:28.817399 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:28.816954 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:28.817399 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:28.817064 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:28.817399 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:28.817141 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:28.817399 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:28.817230 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:30.817539 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:30.817500 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:30.817539 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:30.817524 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:30.818068 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:30.817628 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:30.818068 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:30.817640 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:30.818068 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:30.817722 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:30.818068 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:30.817819 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:32.816991 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:32.816948 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:32.817448 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:32.816948 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:32.817448 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:32.817088 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:32.817448 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:32.817113 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:32.817448 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:32.817185 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:32.817448 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:32.817311 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:34.512505 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:34.512470 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:34.512990 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:34.512618 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:34.512990 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:34.512689 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret podName:91a1ec59-ea70-461a-a9be-471315c4c7df nodeName:}" failed. No retries permitted until 2026-07-17 10:52:50.512668156 +0000 UTC m=+34.186183804 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret") pod "global-pull-secret-syncer-6m5q9" (UID: "91a1ec59-ea70-461a-a9be-471315c4c7df") : object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:34.613676 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:34.613638 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:34.613858 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:34.613810 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:34.613915 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:34.613880 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs podName:c9bb4909-4714-4484-96fb-e258962dfd4b nodeName:}" failed. No retries permitted until 2026-07-17 10:52:50.61386002 +0000 UTC m=+34.287375651 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs") pod "network-metrics-daemon-sqv6x" (UID: "c9bb4909-4714-4484-96fb-e258962dfd4b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:34.714765 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:34.714727 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97m5l\" (UniqueName: \"kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l\") pod \"network-check-target-9p5ns\" (UID: \"5041937f-62cc-401c-9c7e-3a15f4a6e88c\") " pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:34.714938 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:34.714907 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 10:52:34.714938 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:34.714935 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 10:52:34.715018 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:34.714948 2551 projected.go:196] Error preparing data for projected volume kube-api-access-97m5l for pod openshift-network-diagnostics/network-check-target-9p5ns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:34.715018 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:34.715002 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l podName:5041937f-62cc-401c-9c7e-3a15f4a6e88c nodeName:}" failed. No retries permitted until 2026-07-17 10:52:50.714986834 +0000 UTC m=+34.388502463 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-97m5l" (UniqueName: "kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l") pod "network-check-target-9p5ns" (UID: "5041937f-62cc-401c-9c7e-3a15f4a6e88c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:34.817784 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:34.817698 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:34.817936 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:34.817694 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:34.817936 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:34.817823 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:34.817936 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:34.817705 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:34.817936 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:34.817916 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:34.818128 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:34.817956 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:36.818061 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:36.818008 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:36.818567 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:36.818126 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:36.818567 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:36.818145 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:36.818567 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:36.818123 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:36.818567 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:36.818238 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:36.818567 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:36.818351 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:38.816993 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:38.816961 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:38.817523 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:38.816959 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:38.817523 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:38.816981 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:38.817523 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:38.817125 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:38.817523 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:38.817153 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:38.817523 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:38.817221 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:39.954916 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.954525 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffbvd" event={"ID":"339c2065-6a0f-4b69-b150-b18181ebb1f8","Type":"ContainerStarted","Data":"7a9dacd2f19f2085f1a381ab9db7c87822ca86b0a68385ed42f7045d47647f78"} Jul 17 10:52:39.955986 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.955962 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lslhc" event={"ID":"8cceb304-a674-42a1-9620-3d940dc45f70","Type":"ContainerStarted","Data":"6f3d258676d94446cc95019c35a6a1bd2a0f75d08e8504b89ebebab3a5a5cb8d"} Jul 17 10:52:39.957257 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.957233 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ms4m6" event={"ID":"eb6a26fe-e2ba-4d2c-b5e3-bbca1029dfbd","Type":"ContainerStarted","Data":"b8294c1660facaa92ae021b9ba063469a7b463ea90368b2401fdac22a3b81c56"} Jul 17 10:52:39.958446 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.958428 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8qjkv" event={"ID":"88f00775-a242-4da4-bfc9-c73a05c78e24","Type":"ContainerStarted","Data":"7b674ef104ae42478c0ded1982fc2efe7b101f6a12f57cc5808a7981086bf033"} Jul 17 10:52:39.959785 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.959766 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bsl8q" event={"ID":"5e2e9c83-f45d-44d7-b654-fd4187164f25","Type":"ContainerStarted","Data":"f3f4ec9a1c7c76fdc7db187c9c27191afff2796fbd99fee9172d09a2c34ac248"} Jul 17 10:52:39.961014 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.960994 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" event={"ID":"bbc3b057-6133-4198-9876-fd12469991a2","Type":"ContainerStarted","Data":"fb380256f10d35f7fd0a4bd135afa231eb7610a7e778cf33f945dcc344dcd0c3"} Jul 17 10:52:39.962251 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.962231 2551 generic.go:356] "Generic (PLEG): container finished" podID="dff2a623-d2b3-4e48-830a-0d479108fa15" containerID="1491fdb724a83e30ac32c0831570de25f27fc3fe65b2fe27e1f8405a1e0496d9" exitCode=0 Jul 17 10:52:39.962342 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.962295 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4j4j7" event={"ID":"dff2a623-d2b3-4e48-830a-0d479108fa15","Type":"ContainerDied","Data":"1491fdb724a83e30ac32c0831570de25f27fc3fe65b2fe27e1f8405a1e0496d9"} Jul 17 10:52:39.964976 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.964957 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" event={"ID":"bbb466e6-585e-4eb8-8409-9839b0a98861","Type":"ContainerStarted","Data":"09268c9c0f9f74977f1a3f9a9c85a2c59ec84f882d4bd343a41b09e58df1ee51"} Jul 17 10:52:39.965079 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.965012 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" event={"ID":"bbb466e6-585e-4eb8-8409-9839b0a98861","Type":"ContainerStarted","Data":"7b29bc3d6eb33797d8871d507ddc0c14a24952a9ad3737c7b871214f11100c2d"} Jul 17 10:52:39.965079 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.965047 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" event={"ID":"bbb466e6-585e-4eb8-8409-9839b0a98861","Type":"ContainerStarted","Data":"c387140b4658ec570460bb0f1d2d42b700bcece586a279e0035ddf47108fd422"} Jul 17 10:52:39.965079 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.965076 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" event={"ID":"bbb466e6-585e-4eb8-8409-9839b0a98861","Type":"ContainerStarted","Data":"02d0190b0770f63113430f777baa109883b935de55c6615c318cbd9636eb2504"} Jul 17 10:52:39.965184 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.965114 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" event={"ID":"bbb466e6-585e-4eb8-8409-9839b0a98861","Type":"ContainerStarted","Data":"0c8953142a02590aaea3920e86bab0dbb313446a0a2874f3c72d199ee289657c"} Jul 17 10:52:39.965184 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.965142 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" event={"ID":"bbb466e6-585e-4eb8-8409-9839b0a98861","Type":"ContainerStarted","Data":"02fcd121f0c9b50deb3adb5508e16927108f5f3a134be0f0c570fa8ca01eea17"} Jul 17 10:52:39.985758 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.985708 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-42.ec2.internal" podStartSLOduration=21.985679431 podStartE2EDuration="21.985679431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:52:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:21.932441163 +0000 UTC m=+5.605956817" watchObservedRunningTime="2026-07-17 10:52:39.985679431 +0000 UTC m=+23.659195077" Jul 17 10:52:39.986321 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:39.986294 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ffbvd" podStartSLOduration=4.2276313 podStartE2EDuration="22.98628337s" totalImagesPullingTime="18.75865207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:39.985834342 +0000 UTC m=+23.659349986" watchObservedRunningTime="2026-07-17 10:52:39.98628337 +0000 UTC m=+23.659799027" Jul 17 10:52:40.060804 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.060754 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8qjkv" podStartSLOduration=13.40273593 podStartE2EDuration="24.060738266s" totalImagesPullingTime="10.658002336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:52:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:40.044860646 +0000 UTC m=+23.718376299" watchObservedRunningTime="2026-07-17 10:52:40.060738266 +0000 UTC m=+23.734253918" Jul 17 10:52:40.081276 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.081230 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bsl8q" podStartSLOduration=4.294417469 podStartE2EDuration="23.081216982s" totalImagesPullingTime="18.786799513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:40.060454003 +0000 UTC m=+23.733969654" watchObservedRunningTime="2026-07-17 10:52:40.081216982 +0000 UTC m=+23.754732715" Jul 17 10:52:40.081423 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.081387 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ms4m6" podStartSLOduration=5.259337972 podStartE2EDuration="24.081382114s" totalImagesPullingTime="18.822044142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:52:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:40.080931686 +0000 UTC m=+23.754447338" watchObservedRunningTime="2026-07-17 10:52:40.081382114 +0000 UTC m=+23.754897766" Jul 17 10:52:40.105302 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.105213 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lslhc" podStartSLOduration=4.34400236 podStartE2EDuration="23.105196546s" totalImagesPullingTime="18.761194186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:40.105132885 +0000 UTC m=+23.778648537" watchObservedRunningTime="2026-07-17 10:52:40.105196546 +0000 UTC m=+23.778712198" Jul 17 10:52:40.120608 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.120585 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:40.271659 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.271636 2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 17 10:52:40.594513 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.594476 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:40.595033 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.595016 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:40.817139 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.817103 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:40.817139 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.817132 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:40.817482 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.817103 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:40.817482 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:40.817246 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:40.817482 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:40.817379 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:40.817670 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:40.817490 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:40.847065 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.846939 2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T10:52:40.271652553Z","UUID":"69cb81fb-f628-43b6-96da-06cb513f76cb","Handler":null,"Name":"","Endpoint":""} Jul 17 10:52:40.848381 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.848364 2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 17 10:52:40.848474 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.848402 2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 17 10:52:40.969410 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.969376 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" event={"ID":"bbc3b057-6133-4198-9876-fd12469991a2","Type":"ContainerStarted","Data":"be01e0ff2a6b6eb46f5a64cec1c6e6ebae8471042b9eb48553bcda9c402fc777"} Jul 17 10:52:40.971000 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.970975 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mzjpx" event={"ID":"3c452207-798d-430e-8045-7548762dcbc2","Type":"ContainerStarted","Data":"587f5ea0eee6220d4238a3009f5c1a53d367e565884bf0815424cca62ab5b6f9"} Jul 17 10:52:40.971894 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:40.971876 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ffbvd" Jul 17 10:52:41.012036 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:41.011983 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mzjpx" podStartSLOduration=5.225243746 podStartE2EDuration="24.011961312s" totalImagesPullingTime="18.786717566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:40.990851689 +0000 UTC m=+24.664367366" watchObservedRunningTime="2026-07-17 10:52:41.011961312 +0000 UTC m=+24.685476962" Jul 17 10:52:41.977768 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:41.977677 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" event={"ID":"bbb466e6-585e-4eb8-8409-9839b0a98861","Type":"ContainerStarted","Data":"78b5e4b5d70bf4c5596cc11ce7673c797de9b5c850ac096191e6cb27334ce5bb"} Jul 17 10:52:41.979900 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:41.979866 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" event={"ID":"bbc3b057-6133-4198-9876-fd12469991a2","Type":"ContainerStarted","Data":"ecc59961bd3e232739e7e1884e4a585caefb80a86b930220b2b69bcd06796cb4"} Jul 17 10:52:41.999910 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:41.999851 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7kml5" podStartSLOduration=5.182577156 podStartE2EDuration="25.999832061s" totalImagesPullingTime="20.817254905s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:52:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:41.998844625 +0000 UTC m=+25.672360277" watchObservedRunningTime="2026-07-17 10:52:41.999832061 +0000 UTC m=+25.673347713" Jul 17 10:52:42.817729 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:42.817448 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:42.817911 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:42.817449 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:42.818028 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:42.817509 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:42.818092 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:42.818018 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:42.818146 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:42.817852 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:42.818146 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:42.818117 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:44.817643 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:44.817606 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:44.818145 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:44.817607 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:44.818145 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:44.817732 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:44.818145 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:44.817789 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:44.818145 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:44.817610 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:44.818145 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:44.817880 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:45.989984 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:45.989709 2551 generic.go:356] "Generic (PLEG): container finished" podID="dff2a623-d2b3-4e48-830a-0d479108fa15" containerID="5acf8c8a64a3028128526bad4859d1f04ad60a18318cf83b1b631b1504862f93" exitCode=0 Jul 17 10:52:45.990907 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:45.989767 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4j4j7" event={"ID":"dff2a623-d2b3-4e48-830a-0d479108fa15","Type":"ContainerDied","Data":"5acf8c8a64a3028128526bad4859d1f04ad60a18318cf83b1b631b1504862f93"} Jul 17 10:52:45.993890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:45.993863 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" event={"ID":"bbb466e6-585e-4eb8-8409-9839b0a98861","Type":"ContainerStarted","Data":"c0f5cef1979832e670677874aca1810e95f2e5c75d7e3b679dfaa01b9fb3c6a3"} Jul 17 10:52:45.994134 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:45.994119 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:46.009599 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:46.009532 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:46.047854 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:46.047802 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" podStartSLOduration=11.220551241 podStartE2EDuration="30.047785507s" totalImagesPullingTime="18.827234266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:52:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:46.047299026 +0000 UTC m=+29.720814689" watchObservedRunningTime="2026-07-17 10:52:46.047785507 +0000 UTC m=+29.721301159" Jul 17 10:52:46.818133 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:46.818098 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:46.818349 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:46.818221 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:46.818349 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:46.818248 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:46.818438 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:46.818370 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:46.818438 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:46.818407 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:46.818523 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:46.818507 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:46.998125 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:46.998093 2551 generic.go:356] "Generic (PLEG): container finished" podID="dff2a623-d2b3-4e48-830a-0d479108fa15" containerID="45d6f6047bf4d46071adaef352d7c6517b9b08caf2c4259ce6a8e9e7963f1075" exitCode=0 Jul 17 10:52:46.998539 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:46.998175 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4j4j7" event={"ID":"dff2a623-d2b3-4e48-830a-0d479108fa15","Type":"ContainerDied","Data":"45d6f6047bf4d46071adaef352d7c6517b9b08caf2c4259ce6a8e9e7963f1075"} Jul 17 10:52:46.998621 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:46.998596 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:46.998674 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:46.998661 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:47.014690 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:47.014665 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:52:47.184421 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:47.184350 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6m5q9"] Jul 17 10:52:47.184577 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:47.184454 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:47.184577 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:47.184530 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:47.187296 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:47.187270 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9p5ns"] Jul 17 10:52:47.187428 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:47.187379 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:47.187493 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:47.187473 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:47.187913 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:47.187891 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sqv6x"] Jul 17 10:52:47.187994 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:47.187976 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:47.188094 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:47.188073 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:48.816736 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:48.816700 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:48.817288 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:48.816704 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:48.817288 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:48.816833 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:48.817288 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:48.816714 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:48.817288 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:48.816879 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:48.817288 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:48.816955 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:49.004438 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:49.004401 2551 generic.go:356] "Generic (PLEG): container finished" podID="dff2a623-d2b3-4e48-830a-0d479108fa15" containerID="b68efd63603cad43abedf9c7e62c9b1ac348f277bdd5de97aa3490c9ca0390ca" exitCode=0 Jul 17 10:52:49.004613 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:49.004490 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4j4j7" event={"ID":"dff2a623-d2b3-4e48-830a-0d479108fa15","Type":"ContainerDied","Data":"b68efd63603cad43abedf9c7e62c9b1ac348f277bdd5de97aa3490c9ca0390ca"} Jul 17 10:52:50.378814 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:50.378783 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:50.464315 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:50.464277 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:50.527376 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:50.527343 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:50.527578 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:50.527455 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:50.527578 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:50.527511 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret podName:91a1ec59-ea70-461a-a9be-471315c4c7df nodeName:}" failed. No retries permitted until 2026-07-17 10:53:22.527497281 +0000 UTC m=+66.201012914 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret") pod "global-pull-secret-syncer-6m5q9" (UID: "91a1ec59-ea70-461a-a9be-471315c4c7df") : object "kube-system"/"original-pull-secret" not registered Jul 17 10:52:50.628115 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:50.628042 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:50.628273 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:50.628181 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:50.628273 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:50.628236 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs podName:c9bb4909-4714-4484-96fb-e258962dfd4b nodeName:}" failed. No retries permitted until 2026-07-17 10:53:22.628222166 +0000 UTC m=+66.301737809 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs") pod "network-metrics-daemon-sqv6x" (UID: "c9bb4909-4714-4484-96fb-e258962dfd4b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 10:52:50.728675 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:50.728630 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97m5l\" (UniqueName: \"kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l\") pod \"network-check-target-9p5ns\" (UID: \"5041937f-62cc-401c-9c7e-3a15f4a6e88c\") " pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:50.728908 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:50.728784 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 10:52:50.728908 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:50.728807 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 10:52:50.728908 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:50.728816 2551 projected.go:196] Error preparing data for projected volume kube-api-access-97m5l for pod openshift-network-diagnostics/network-check-target-9p5ns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:50.728908 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:50.728867 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l podName:5041937f-62cc-401c-9c7e-3a15f4a6e88c nodeName:}" failed. No retries permitted until 2026-07-17 10:53:22.72885352 +0000 UTC m=+66.402369150 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-97m5l" (UniqueName: "kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l") pod "network-check-target-9p5ns" (UID: "5041937f-62cc-401c-9c7e-3a15f4a6e88c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 10:52:50.816992 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:50.816954 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:50.817231 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:50.816954 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:50.817231 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:50.817075 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9p5ns" podUID="5041937f-62cc-401c-9c7e-3a15f4a6e88c" Jul 17 10:52:50.817231 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:50.817166 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sqv6x" podUID="c9bb4909-4714-4484-96fb-e258962dfd4b" Jul 17 10:52:50.817231 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:50.816954 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:50.817439 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:50.817258 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6m5q9" podUID="91a1ec59-ea70-461a-a9be-471315c4c7df" Jul 17 10:52:52.647381 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.647307 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-42.ec2.internal" event="NodeReady" Jul 17 10:52:52.647972 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.647464 2551 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 17 10:52:52.695527 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.695488 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-66wkj"] Jul 17 10:52:52.701342 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.701316 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lw2nf"] Jul 17 10:52:52.701474 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.701436 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.703855 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.703831 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 17 10:52:52.703987 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.703970 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 17 10:52:52.704053 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.703981 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dnhpd\"" Jul 17 10:52:52.705352 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.705333 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lw2nf" Jul 17 10:52:52.709802 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.709781 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-25nnq\"" Jul 17 10:52:52.709921 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.709839 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 17 10:52:52.710031 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.709917 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 17 10:52:52.711119 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.710704 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 17 10:52:52.711119 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.710994 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lw2nf"] Jul 17 10:52:52.712236 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.712152 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-66wkj"] Jul 17 10:52:52.811656 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.811619 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:52.817520 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.817495 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:52:52.817671 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.817502 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:52:52.817671 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.817644 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:52:52.820800 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.820442 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 10:52:52.820800 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.820456 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 10:52:52.820800 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.820447 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 10:52:52.820800 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.820526 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gt9xr\"" Jul 17 10:52:52.820800 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.820562 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 10:52:52.821098 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.820836 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rk769\"" Jul 17 10:52:52.842317 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.842295 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38ad94cb-fac6-47f0-9024-9d1a71f16caf-cert\") pod \"ingress-canary-lw2nf\" (UID: \"38ad94cb-fac6-47f0-9024-9d1a71f16caf\") " pod="openshift-ingress-canary/ingress-canary-lw2nf" Jul 17 10:52:52.842420 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.842328 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46c254d9-45eb-404e-a08e-60c8bbf2314e-tmp-dir\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.842420 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.842394 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.842536 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.842420 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l84z6\" (UniqueName: \"kubernetes.io/projected/46c254d9-45eb-404e-a08e-60c8bbf2314e-kube-api-access-l84z6\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.842536 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.842481 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v89jk\" (UniqueName: \"kubernetes.io/projected/38ad94cb-fac6-47f0-9024-9d1a71f16caf-kube-api-access-v89jk\") pod \"ingress-canary-lw2nf\" (UID: \"38ad94cb-fac6-47f0-9024-9d1a71f16caf\") " pod="openshift-ingress-canary/ingress-canary-lw2nf" Jul 17 10:52:52.842536 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.842507 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46c254d9-45eb-404e-a08e-60c8bbf2314e-config-volume\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.865289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.865256 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:52.943877 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.943790 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38ad94cb-fac6-47f0-9024-9d1a71f16caf-cert\") pod \"ingress-canary-lw2nf\" (UID: \"38ad94cb-fac6-47f0-9024-9d1a71f16caf\") " pod="openshift-ingress-canary/ingress-canary-lw2nf" Jul 17 10:52:52.943877 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.943840 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46c254d9-45eb-404e-a08e-60c8bbf2314e-tmp-dir\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.944111 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.943917 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.944164 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.944105 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l84z6\" (UniqueName: \"kubernetes.io/projected/46c254d9-45eb-404e-a08e-60c8bbf2314e-kube-api-access-l84z6\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.944220 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.944176 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v89jk\" (UniqueName: \"kubernetes.io/projected/38ad94cb-fac6-47f0-9024-9d1a71f16caf-kube-api-access-v89jk\") pod \"ingress-canary-lw2nf\" (UID: \"38ad94cb-fac6-47f0-9024-9d1a71f16caf\") " pod="openshift-ingress-canary/ingress-canary-lw2nf" Jul 17 10:52:52.944220 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:52.944182 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 10:52:52.944220 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.944203 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46c254d9-45eb-404e-a08e-60c8bbf2314e-config-volume\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.944369 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.944250 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46c254d9-45eb-404e-a08e-60c8bbf2314e-tmp-dir\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.944369 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:52.944257 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls podName:46c254d9-45eb-404e-a08e-60c8bbf2314e nodeName:}" failed. No retries permitted until 2026-07-17 10:52:53.444235215 +0000 UTC m=+37.117750859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls") pod "dns-default-66wkj" (UID: "46c254d9-45eb-404e-a08e-60c8bbf2314e") : secret "dns-default-metrics-tls" not found Jul 17 10:52:52.944891 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.944870 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46c254d9-45eb-404e-a08e-60c8bbf2314e-config-volume\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.948958 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.948934 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38ad94cb-fac6-47f0-9024-9d1a71f16caf-cert\") pod \"ingress-canary-lw2nf\" (UID: \"38ad94cb-fac6-47f0-9024-9d1a71f16caf\") " pod="openshift-ingress-canary/ingress-canary-lw2nf" Jul 17 10:52:52.953258 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.953220 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l84z6\" (UniqueName: \"kubernetes.io/projected/46c254d9-45eb-404e-a08e-60c8bbf2314e-kube-api-access-l84z6\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:52.953398 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:52.953260 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v89jk\" (UniqueName: \"kubernetes.io/projected/38ad94cb-fac6-47f0-9024-9d1a71f16caf-kube-api-access-v89jk\") pod \"ingress-canary-lw2nf\" (UID: \"38ad94cb-fac6-47f0-9024-9d1a71f16caf\") " pod="openshift-ingress-canary/ingress-canary-lw2nf" Jul 17 10:52:53.020634 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:53.020602 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lw2nf" Jul 17 10:52:53.322688 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:53.322657 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lw2nf"] Jul 17 10:52:53.325908 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:52:53.325881 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38ad94cb_fac6_47f0_9024_9d1a71f16caf.slice/crio-b2cf93712b0dac9fac42b7b3aeec6993bfaa7a9924171dae1a8e0e03229d2878 WatchSource:0}: Error finding container b2cf93712b0dac9fac42b7b3aeec6993bfaa7a9924171dae1a8e0e03229d2878: Status 404 returned error can't find the container with id b2cf93712b0dac9fac42b7b3aeec6993bfaa7a9924171dae1a8e0e03229d2878 Jul 17 10:52:53.447862 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:53.447764 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:53.447974 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:53.447907 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 10:52:53.447974 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:53.447974 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls podName:46c254d9-45eb-404e-a08e-60c8bbf2314e nodeName:}" failed. No retries permitted until 2026-07-17 10:52:54.447958228 +0000 UTC m=+38.121473859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls") pod "dns-default-66wkj" (UID: "46c254d9-45eb-404e-a08e-60c8bbf2314e") : secret "dns-default-metrics-tls" not found Jul 17 10:52:54.021811 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:54.021772 2551 generic.go:356] "Generic (PLEG): container finished" podID="dff2a623-d2b3-4e48-830a-0d479108fa15" containerID="4b0e7b5bfa5f3882ba1312ad7fb914b676a8381ffb144277abd4173efa24ebde" exitCode=0 Jul 17 10:52:54.022364 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:54.021870 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4j4j7" event={"ID":"dff2a623-d2b3-4e48-830a-0d479108fa15","Type":"ContainerDied","Data":"4b0e7b5bfa5f3882ba1312ad7fb914b676a8381ffb144277abd4173efa24ebde"} Jul 17 10:52:54.023116 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:54.023089 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lw2nf" event={"ID":"38ad94cb-fac6-47f0-9024-9d1a71f16caf","Type":"ContainerStarted","Data":"b2cf93712b0dac9fac42b7b3aeec6993bfaa7a9924171dae1a8e0e03229d2878"} Jul 17 10:52:54.455804 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:54.455761 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:54.455976 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:54.455935 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 10:52:54.456041 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:54.456004 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls podName:46c254d9-45eb-404e-a08e-60c8bbf2314e nodeName:}" failed. No retries permitted until 2026-07-17 10:52:56.455987982 +0000 UTC m=+40.129503611 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls") pod "dns-default-66wkj" (UID: "46c254d9-45eb-404e-a08e-60c8bbf2314e") : secret "dns-default-metrics-tls" not found Jul 17 10:52:54.663901 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:54.663863 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:54.763534 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:54.763495 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:55.029252 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:55.029174 2551 generic.go:356] "Generic (PLEG): container finished" podID="dff2a623-d2b3-4e48-830a-0d479108fa15" containerID="d4ddb205771a576bd7224fbf63708b625022fc0accc3235f5df80bc87d9701e2" exitCode=0 Jul 17 10:52:55.029252 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:55.029232 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4j4j7" event={"ID":"dff2a623-d2b3-4e48-830a-0d479108fa15","Type":"ContainerDied","Data":"d4ddb205771a576bd7224fbf63708b625022fc0accc3235f5df80bc87d9701e2"} Jul 17 10:52:55.384119 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:55.384092 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:55.484598 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:55.484543 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:55.903604 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:55.903360 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:55.967619 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:55.967580 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:56.034259 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:56.034225 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4j4j7" event={"ID":"dff2a623-d2b3-4e48-830a-0d479108fa15","Type":"ContainerStarted","Data":"2345c1620e3e549af96ab40cfd5e1a53a1e80b0c262ebe51265f922264721df9"} Jul 17 10:52:56.035628 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:56.035603 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lw2nf" event={"ID":"38ad94cb-fac6-47f0-9024-9d1a71f16caf","Type":"ContainerStarted","Data":"3540a20f23495491f3710a82a0f7eef525d0432b2b685a3ac224e20344243cc0"} Jul 17 10:52:56.057863 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:56.057804 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4j4j7" podStartSLOduration=9.298928773 podStartE2EDuration="39.057786603s" totalImagesPullingTime="29.75885783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:52:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:56.057280389 +0000 UTC m=+39.730796040" watchObservedRunningTime="2026-07-17 10:52:56.057786603 +0000 UTC m=+39.731302256" Jul 17 10:52:56.072830 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:56.072772 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lw2nf" podStartSLOduration=1.773949255 podStartE2EDuration="4.072754608s" totalImagesPullingTime="2.298805353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:52:56.071953406 +0000 UTC m=+39.745469058" watchObservedRunningTime="2026-07-17 10:52:56.072754608 +0000 UTC m=+39.746270261" Jul 17 10:52:56.184575 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:56.184471 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:56.239712 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:56.239682 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:52:56.471789 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:52:56.471704 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:52:56.471928 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:56.471854 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 10:52:56.471928 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:52:56.471917 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls podName:46c254d9-45eb-404e-a08e-60c8bbf2314e nodeName:}" failed. No retries permitted until 2026-07-17 10:53:00.471902286 +0000 UTC m=+44.145417916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls") pod "dns-default-66wkj" (UID: "46c254d9-45eb-404e-a08e-60c8bbf2314e") : secret "dns-default-metrics-tls" not found Jul 17 10:53:00.459328 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:00.459283 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:53:00.504328 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:00.504301 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:53:00.504471 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:00.504430 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 10:53:00.504508 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:00.504484 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls podName:46c254d9-45eb-404e-a08e-60c8bbf2314e nodeName:}" failed. No retries permitted until 2026-07-17 10:53:08.504469289 +0000 UTC m=+52.177984919 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls") pod "dns-default-66wkj" (UID: "46c254d9-45eb-404e-a08e-60c8bbf2314e") : secret "dns-default-metrics-tls" not found Jul 17 10:53:00.559737 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:00.559701 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:53:00.844102 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:00.844068 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" limit=1024 Jul 17 10:53:08.552184 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:08.552148 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:53:08.552613 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:08.552302 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 10:53:08.552613 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:08.552365 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls podName:46c254d9-45eb-404e-a08e-60c8bbf2314e nodeName:}" failed. No retries permitted until 2026-07-17 10:53:24.552349702 +0000 UTC m=+68.225865332 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls") pod "dns-default-66wkj" (UID: "46c254d9-45eb-404e-a08e-60c8bbf2314e") : secret "dns-default-metrics-tls" not found Jul 17 10:53:15.253838 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.253804 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh"] Jul 17 10:53:15.255814 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.255797 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:15.259072 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.259049 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 17 10:53:15.259072 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.259062 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9dnn6\"" Jul 17 10:53:15.259262 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.259049 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 17 10:53:15.259262 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.259053 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 17 10:53:15.267080 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.267061 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh"] Jul 17 10:53:15.299940 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.299890 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkr68\" (UniqueName: \"kubernetes.io/projected/f41711b4-31e1-48c3-9f7f-1c9714d59b06-kube-api-access-xkr68\") pod \"cluster-samples-operator-79b7c55f87-fl9vh\" (UID: \"f41711b4-31e1-48c3-9f7f-1c9714d59b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:15.299940 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.299947 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fl9vh\" (UID: \"f41711b4-31e1-48c3-9f7f-1c9714d59b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:15.377645 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.377614 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-lxp5b"] Jul 17 10:53:15.379507 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.379491 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.382279 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.382259 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 17 10:53:15.382510 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.382490 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 17 10:53:15.382611 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.382499 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 17 10:53:15.382611 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.382570 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-cdg22\"" Jul 17 10:53:15.382611 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.382581 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 17 10:53:15.393076 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.393051 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 17 10:53:15.397287 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.397264 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-lxp5b"] Jul 17 10:53:15.400254 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.400235 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c351f21-c45a-4d18-939d-5f4159aad6fb-service-ca-bundle\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.400327 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.400262 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c351f21-c45a-4d18-939d-5f4159aad6fb-serving-cert\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.400327 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.400288 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c351f21-c45a-4d18-939d-5f4159aad6fb-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.400453 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.400425 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkr68\" (UniqueName: \"kubernetes.io/projected/f41711b4-31e1-48c3-9f7f-1c9714d59b06-kube-api-access-xkr68\") pod \"cluster-samples-operator-79b7c55f87-fl9vh\" (UID: \"f41711b4-31e1-48c3-9f7f-1c9714d59b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:15.400594 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.400466 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb6hn\" (UniqueName: \"kubernetes.io/projected/6c351f21-c45a-4d18-939d-5f4159aad6fb-kube-api-access-hb6hn\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.400594 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.400543 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fl9vh\" (UID: \"f41711b4-31e1-48c3-9f7f-1c9714d59b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:15.400594 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.400587 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c351f21-c45a-4d18-939d-5f4159aad6fb-tmp\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.400774 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.400603 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6c351f21-c45a-4d18-939d-5f4159aad6fb-snapshots\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.400774 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:15.400734 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 10:53:15.400847 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:15.400806 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls podName:f41711b4-31e1-48c3-9f7f-1c9714d59b06 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:15.900787769 +0000 UTC m=+59.574303417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fl9vh" (UID: "f41711b4-31e1-48c3-9f7f-1c9714d59b06") : secret "samples-operator-tls" not found Jul 17 10:53:15.420854 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.420832 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkr68\" (UniqueName: \"kubernetes.io/projected/f41711b4-31e1-48c3-9f7f-1c9714d59b06-kube-api-access-xkr68\") pod \"cluster-samples-operator-79b7c55f87-fl9vh\" (UID: \"f41711b4-31e1-48c3-9f7f-1c9714d59b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:15.493531 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.493496 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-r6gsp"] Jul 17 10:53:15.495398 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.495382 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.501154 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.501129 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb6hn\" (UniqueName: \"kubernetes.io/projected/6c351f21-c45a-4d18-939d-5f4159aad6fb-kube-api-access-hb6hn\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.501289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.501224 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c351f21-c45a-4d18-939d-5f4159aad6fb-tmp\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.501289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.501257 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6c351f21-c45a-4d18-939d-5f4159aad6fb-snapshots\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.501503 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.501318 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c351f21-c45a-4d18-939d-5f4159aad6fb-service-ca-bundle\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.501503 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.501345 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c351f21-c45a-4d18-939d-5f4159aad6fb-serving-cert\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.501503 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.501415 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c351f21-c45a-4d18-939d-5f4159aad6fb-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.501771 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.501715 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c351f21-c45a-4d18-939d-5f4159aad6fb-tmp\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.502023 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.501989 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c351f21-c45a-4d18-939d-5f4159aad6fb-service-ca-bundle\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.502354 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.502335 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c351f21-c45a-4d18-939d-5f4159aad6fb-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.502453 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.502353 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6c351f21-c45a-4d18-939d-5f4159aad6fb-snapshots\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.503697 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.503678 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c351f21-c45a-4d18-939d-5f4159aad6fb-serving-cert\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.514530 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.514486 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 17 10:53:15.526864 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.526845 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b5c697c4d-rz846"] Jul 17 10:53:15.528807 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.528793 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.543752 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.543729 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 17 10:53:15.544384 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.544372 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-96lrv\"" Jul 17 10:53:15.544563 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.544538 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 17 10:53:15.544610 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.544581 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 17 10:53:15.545104 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.545089 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 17 10:53:15.546574 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.546560 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 17 10:53:15.547402 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.547390 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gcsz5\"" Jul 17 10:53:15.554704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.554678 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 17 10:53:15.555757 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.555729 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 17 10:53:15.555844 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.555797 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 17 10:53:15.558232 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.558212 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-r6gsp"] Jul 17 10:53:15.559610 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.559593 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb6hn\" (UniqueName: \"kubernetes.io/projected/6c351f21-c45a-4d18-939d-5f4159aad6fb-kube-api-access-hb6hn\") pod \"insights-operator-555d9d7885-lxp5b\" (UID: \"6c351f21-c45a-4d18-939d-5f4159aad6fb\") " pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.567167 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.567144 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b5c697c4d-rz846"] Jul 17 10:53:15.601851 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.601825 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55a709f2-51ec-4d79-923e-198fd980ba4b-serving-cert\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.602006 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.601983 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-installation-pull-secrets\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.602075 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.602035 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/55a709f2-51ec-4d79-923e-198fd980ba4b-trusted-ca\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.602075 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.602065 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-image-registry-private-configuration\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.602174 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.602107 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55a709f2-51ec-4d79-923e-198fd980ba4b-config\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.602174 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.602132 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-bound-sa-token\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.602263 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.602206 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgskg\" (UniqueName: \"kubernetes.io/projected/55a709f2-51ec-4d79-923e-198fd980ba4b-kube-api-access-kgskg\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.602263 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.602239 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.602345 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.602260 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b8b6e43-da60-423a-9108-fda424fade1a-ca-trust-extracted\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.602345 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.602284 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-trusted-ca\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.602345 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.602304 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzfwc\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-kube-api-access-hzfwc\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.602345 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.602327 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-registry-certificates\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.688462 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.688432 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-lxp5b" Jul 17 10:53:15.702882 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.702855 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-installation-pull-secrets\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.702998 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.702897 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/55a709f2-51ec-4d79-923e-198fd980ba4b-trusted-ca\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.702998 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.702925 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-image-registry-private-configuration\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.702998 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.702968 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55a709f2-51ec-4d79-923e-198fd980ba4b-config\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.702998 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.702993 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-bound-sa-token\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.703167 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.703041 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgskg\" (UniqueName: \"kubernetes.io/projected/55a709f2-51ec-4d79-923e-198fd980ba4b-kube-api-access-kgskg\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.703167 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.703077 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.703167 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.703099 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b8b6e43-da60-423a-9108-fda424fade1a-ca-trust-extracted\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.703312 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:15.703210 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 10:53:15.703312 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:15.703233 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b5c697c4d-rz846: secret "image-registry-tls" not found Jul 17 10:53:15.703312 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:15.703305 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls podName:4b8b6e43-da60-423a-9108-fda424fade1a nodeName:}" failed. No retries permitted until 2026-07-17 10:53:16.203284787 +0000 UTC m=+59.876800440 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls") pod "image-registry-6b5c697c4d-rz846" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a") : secret "image-registry-tls" not found Jul 17 10:53:15.703481 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.703460 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b8b6e43-da60-423a-9108-fda424fade1a-ca-trust-extracted\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.703613 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.703590 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-trusted-ca\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.703741 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.703630 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzfwc\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-kube-api-access-hzfwc\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.703741 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.703668 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-registry-certificates\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.703861 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.703745 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55a709f2-51ec-4d79-923e-198fd980ba4b-serving-cert\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.704051 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.704030 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/55a709f2-51ec-4d79-923e-198fd980ba4b-trusted-ca\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.704413 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.704388 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-registry-certificates\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.704597 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.704575 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55a709f2-51ec-4d79-923e-198fd980ba4b-config\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.704597 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.704586 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-trusted-ca\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.705668 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.705649 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-image-registry-private-configuration\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.705764 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.705749 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-installation-pull-secrets\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.705995 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.705981 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55a709f2-51ec-4d79-923e-198fd980ba4b-serving-cert\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.718390 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.718363 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzfwc\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-kube-api-access-hzfwc\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.718764 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.718744 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-bound-sa-token\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:15.719213 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.719196 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgskg\" (UniqueName: \"kubernetes.io/projected/55a709f2-51ec-4d79-923e-198fd980ba4b-kube-api-access-kgskg\") pod \"console-operator-565b7b569d-r6gsp\" (UID: \"55a709f2-51ec-4d79-923e-198fd980ba4b\") " pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.804534 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.804501 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:15.812204 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.812176 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-lxp5b"] Jul 17 10:53:15.815667 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:15.815639 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c351f21_c45a_4d18_939d_5f4159aad6fb.slice/crio-45d65ec9f6c607d47dd89dbc94749edf09297ca94525dc191c3a6ba51857cb15 WatchSource:0}: Error finding container 45d65ec9f6c607d47dd89dbc94749edf09297ca94525dc191c3a6ba51857cb15: Status 404 returned error can't find the container with id 45d65ec9f6c607d47dd89dbc94749edf09297ca94525dc191c3a6ba51857cb15 Jul 17 10:53:15.905322 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.905283 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fl9vh\" (UID: \"f41711b4-31e1-48c3-9f7f-1c9714d59b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:15.905464 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:15.905421 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 10:53:15.905510 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:15.905487 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls podName:f41711b4-31e1-48c3-9f7f-1c9714d59b06 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:16.905470141 +0000 UTC m=+60.578985770 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fl9vh" (UID: "f41711b4-31e1-48c3-9f7f-1c9714d59b06") : secret "samples-operator-tls" not found Jul 17 10:53:15.934141 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:15.934112 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-r6gsp"] Jul 17 10:53:15.937109 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:15.937082 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a709f2_51ec_4d79_923e_198fd980ba4b.slice/crio-904921e741929e4f18860d334572863b2737e6fc72d08806bbe976a6e704d3de WatchSource:0}: Error finding container 904921e741929e4f18860d334572863b2737e6fc72d08806bbe976a6e704d3de: Status 404 returned error can't find the container with id 904921e741929e4f18860d334572863b2737e6fc72d08806bbe976a6e704d3de Jul 17 10:53:16.079992 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:16.079896 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lxp5b" event={"ID":"6c351f21-c45a-4d18-939d-5f4159aad6fb","Type":"ContainerStarted","Data":"45d65ec9f6c607d47dd89dbc94749edf09297ca94525dc191c3a6ba51857cb15"} Jul 17 10:53:16.080924 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:16.080900 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" event={"ID":"55a709f2-51ec-4d79-923e-198fd980ba4b","Type":"ContainerStarted","Data":"904921e741929e4f18860d334572863b2737e6fc72d08806bbe976a6e704d3de"} Jul 17 10:53:16.208704 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:16.208666 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:16.208894 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:16.208852 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 10:53:16.208894 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:16.208874 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b5c697c4d-rz846: secret "image-registry-tls" not found Jul 17 10:53:16.209001 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:16.208953 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls podName:4b8b6e43-da60-423a-9108-fda424fade1a nodeName:}" failed. No retries permitted until 2026-07-17 10:53:17.208929914 +0000 UTC m=+60.882445561 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls") pod "image-registry-6b5c697c4d-rz846" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a") : secret "image-registry-tls" not found Jul 17 10:53:16.916321 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:16.916285 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fl9vh\" (UID: \"f41711b4-31e1-48c3-9f7f-1c9714d59b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:16.916818 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:16.916470 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 10:53:16.916818 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:16.916563 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls podName:f41711b4-31e1-48c3-9f7f-1c9714d59b06 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:18.916529863 +0000 UTC m=+62.590045513 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fl9vh" (UID: "f41711b4-31e1-48c3-9f7f-1c9714d59b06") : secret "samples-operator-tls" not found Jul 17 10:53:17.218917 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:17.218834 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:17.219065 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:17.218946 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 10:53:17.219065 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:17.218958 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b5c697c4d-rz846: secret "image-registry-tls" not found Jul 17 10:53:17.219065 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:17.219004 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls podName:4b8b6e43-da60-423a-9108-fda424fade1a nodeName:}" failed. No retries permitted until 2026-07-17 10:53:19.218991662 +0000 UTC m=+62.892507292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls") pod "image-registry-6b5c697c4d-rz846" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a") : secret "image-registry-tls" not found Jul 17 10:53:18.936624 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:18.936576 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fl9vh\" (UID: \"f41711b4-31e1-48c3-9f7f-1c9714d59b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:18.937158 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:18.936802 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 10:53:18.937158 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:18.936880 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls podName:f41711b4-31e1-48c3-9f7f-1c9714d59b06 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:22.936859488 +0000 UTC m=+66.610375141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fl9vh" (UID: "f41711b4-31e1-48c3-9f7f-1c9714d59b06") : secret "samples-operator-tls" not found Jul 17 10:53:19.015609 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.015580 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vxr2t" Jul 17 10:53:19.089066 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.089024 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lxp5b" event={"ID":"6c351f21-c45a-4d18-939d-5f4159aad6fb","Type":"ContainerStarted","Data":"2762df7347b269f19a45b84a992544080e4692a3879529467905f347cb539827"} Jul 17 10:53:19.112658 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.112598 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-lxp5b" podStartSLOduration=1.829418951 podStartE2EDuration="4.112575346s" totalImagesPullingTime="2.283156395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:53:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:19.111436011 +0000 UTC m=+62.784951664" watchObservedRunningTime="2026-07-17 10:53:19.112575346 +0000 UTC m=+62.786090994" Jul 17 10:53:19.239346 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.239267 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:19.239493 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:19.239407 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 10:53:19.239493 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:19.239422 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b5c697c4d-rz846: secret "image-registry-tls" not found Jul 17 10:53:19.239493 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:19.239479 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls podName:4b8b6e43-da60-423a-9108-fda424fade1a nodeName:}" failed. No retries permitted until 2026-07-17 10:53:23.239462603 +0000 UTC m=+66.912978253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls") pod "image-registry-6b5c697c4d-rz846" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a") : secret "image-registry-tls" not found Jul 17 10:53:19.859985 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.859954 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2l2qk"] Jul 17 10:53:19.863030 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.863002 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:19.865623 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.865601 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 17 10:53:19.865752 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.865693 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 17 10:53:19.865923 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.865902 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-x57x9\"" Jul 17 10:53:19.875105 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.875085 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2l2qk"] Jul 17 10:53:19.945462 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.945433 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/357ff6cd-3d67-44d5-9f4d-638229f579a6-data-volume\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:19.945902 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.945483 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/357ff6cd-3d67-44d5-9f4d-638229f579a6-crio-socket\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:19.945902 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.945507 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:19.945902 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.945693 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zln8\" (UniqueName: \"kubernetes.io/projected/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-api-access-5zln8\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:19.945902 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:19.945730 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:20.046639 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.046598 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/357ff6cd-3d67-44d5-9f4d-638229f579a6-data-volume\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:20.046827 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.046658 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/357ff6cd-3d67-44d5-9f4d-638229f579a6-crio-socket\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:20.046827 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.046677 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:20.046827 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.046726 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zln8\" (UniqueName: \"kubernetes.io/projected/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-api-access-5zln8\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:20.046827 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.046745 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:20.046827 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.046804 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/357ff6cd-3d67-44d5-9f4d-638229f579a6-crio-socket\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:20.046827 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:20.046826 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 10:53:20.047068 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:20.046832 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 10:53:20.047068 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:20.046903 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm podName:357ff6cd-3d67-44d5-9f4d-638229f579a6 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:20.546883404 +0000 UTC m=+64.220399049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2l2qk" (UID: "357ff6cd-3d67-44d5-9f4d-638229f579a6") : configmap "kube-rbac-proxy" not found Jul 17 10:53:20.047068 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:20.046923 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls podName:357ff6cd-3d67-44d5-9f4d-638229f579a6 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:20.54691425 +0000 UTC m=+64.220429881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2l2qk" (UID: "357ff6cd-3d67-44d5-9f4d-638229f579a6") : secret "insights-runtime-extractor-tls" not found Jul 17 10:53:20.047068 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.046937 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/357ff6cd-3d67-44d5-9f4d-638229f579a6-data-volume\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:20.060205 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.060180 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zln8\" (UniqueName: \"kubernetes.io/projected/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-api-access-5zln8\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:20.092807 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.092783 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/0.log" Jul 17 10:53:20.092940 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.092820 2551 generic.go:356] "Generic (PLEG): container finished" podID="55a709f2-51ec-4d79-923e-198fd980ba4b" containerID="2b392515c44293b124cd74a109ba82a1d882b0bfaef7e03d8f33191883055fbc" exitCode=255 Jul 17 10:53:20.092940 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.092853 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" event={"ID":"55a709f2-51ec-4d79-923e-198fd980ba4b","Type":"ContainerDied","Data":"2b392515c44293b124cd74a109ba82a1d882b0bfaef7e03d8f33191883055fbc"} Jul 17 10:53:20.093132 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.093116 2551 scope.go:117] "RemoveContainer" containerID="2b392515c44293b124cd74a109ba82a1d882b0bfaef7e03d8f33191883055fbc" Jul 17 10:53:20.550523 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.550495 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:20.550719 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:20.550607 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:20.550719 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:20.550618 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 10:53:20.550719 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:20.550679 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm podName:357ff6cd-3d67-44d5-9f4d-638229f579a6 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:21.550665393 +0000 UTC m=+65.224181023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2l2qk" (UID: "357ff6cd-3d67-44d5-9f4d-638229f579a6") : configmap "kube-rbac-proxy" not found Jul 17 10:53:20.550719 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:20.550695 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 10:53:20.550871 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:20.550741 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls podName:357ff6cd-3d67-44d5-9f4d-638229f579a6 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:21.550729422 +0000 UTC m=+65.224245056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2l2qk" (UID: "357ff6cd-3d67-44d5-9f4d-638229f579a6") : secret "insights-runtime-extractor-tls" not found Jul 17 10:53:21.096822 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:21.096792 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 10:53:21.097272 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:21.097234 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/0.log" Jul 17 10:53:21.097358 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:21.097297 2551 generic.go:356] "Generic (PLEG): container finished" podID="55a709f2-51ec-4d79-923e-198fd980ba4b" containerID="eea949d5513ffa8edf33987ab0bda1629f8a036c6bfc420acf5f3d31891a4716" exitCode=255 Jul 17 10:53:21.097400 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:21.097378 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" event={"ID":"55a709f2-51ec-4d79-923e-198fd980ba4b","Type":"ContainerDied","Data":"eea949d5513ffa8edf33987ab0bda1629f8a036c6bfc420acf5f3d31891a4716"} Jul 17 10:53:21.097451 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:21.097440 2551 scope.go:117] "RemoveContainer" containerID="2b392515c44293b124cd74a109ba82a1d882b0bfaef7e03d8f33191883055fbc" Jul 17 10:53:21.097699 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:21.097684 2551 scope.go:117] "RemoveContainer" containerID="eea949d5513ffa8edf33987ab0bda1629f8a036c6bfc420acf5f3d31891a4716" Jul 17 10:53:21.097881 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:21.097864 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-r6gsp_openshift-console-operator(55a709f2-51ec-4d79-923e-198fd980ba4b)\"" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" podUID="55a709f2-51ec-4d79-923e-198fd980ba4b" Jul 17 10:53:21.559818 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:21.559786 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:21.559987 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:21.559857 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:21.559987 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:21.559954 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 10:53:21.560057 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:21.560003 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm podName:357ff6cd-3d67-44d5-9f4d-638229f579a6 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:23.559988659 +0000 UTC m=+67.233504292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2l2qk" (UID: "357ff6cd-3d67-44d5-9f4d-638229f579a6") : configmap "kube-rbac-proxy" not found Jul 17 10:53:21.560057 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:21.559954 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 10:53:21.560124 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:21.560083 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls podName:357ff6cd-3d67-44d5-9f4d-638229f579a6 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:23.560070476 +0000 UTC m=+67.233586107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2l2qk" (UID: "357ff6cd-3d67-44d5-9f4d-638229f579a6") : secret "insights-runtime-extractor-tls" not found Jul 17 10:53:21.657130 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:21.657103 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bsl8q_5e2e9c83-f45d-44d7-b654-fd4187164f25/dns-node-resolver/0.log" Jul 17 10:53:22.104430 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.104403 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 10:53:22.104843 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.104708 2551 scope.go:117] "RemoveContainer" containerID="eea949d5513ffa8edf33987ab0bda1629f8a036c6bfc420acf5f3d31891a4716" Jul 17 10:53:22.104887 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:22.104852 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-r6gsp_openshift-console-operator(55a709f2-51ec-4d79-923e-198fd980ba4b)\"" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" podUID="55a709f2-51ec-4d79-923e-198fd980ba4b" Jul 17 10:53:22.569089 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.569050 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:53:22.572000 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.571983 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 10:53:22.582291 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.582264 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91a1ec59-ea70-461a-a9be-471315c4c7df-original-pull-secret\") pod \"global-pull-secret-syncer-6m5q9\" (UID: \"91a1ec59-ea70-461a-a9be-471315c4c7df\") " pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:53:22.669780 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.669746 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:53:22.672688 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.672671 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 10:53:22.680798 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:22.680773 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 17 10:53:22.680847 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:22.680834 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs podName:c9bb4909-4714-4484-96fb-e258962dfd4b nodeName:}" failed. No retries permitted until 2026-07-17 10:54:26.680818613 +0000 UTC m=+130.354334243 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs") pod "network-metrics-daemon-sqv6x" (UID: "c9bb4909-4714-4484-96fb-e258962dfd4b") : secret "metrics-daemon-secret" not found Jul 17 10:53:22.770496 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.770463 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97m5l\" (UniqueName: \"kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l\") pod \"network-check-target-9p5ns\" (UID: \"5041937f-62cc-401c-9c7e-3a15f4a6e88c\") " pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:53:22.773635 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.773619 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 10:53:22.784294 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.784267 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 10:53:22.793619 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.793598 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97m5l\" (UniqueName: \"kubernetes.io/projected/5041937f-62cc-401c-9c7e-3a15f4a6e88c-kube-api-access-97m5l\") pod \"network-check-target-9p5ns\" (UID: \"5041937f-62cc-401c-9c7e-3a15f4a6e88c\") " pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:53:22.829647 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.829573 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6m5q9" Jul 17 10:53:22.839033 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.839009 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gt9xr\"" Jul 17 10:53:22.847178 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.847152 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:53:22.854172 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.854151 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8qjkv_88f00775-a242-4da4-bfc9-c73a05c78e24/node-ca/0.log" Jul 17 10:53:22.960482 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.960454 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6m5q9"] Jul 17 10:53:22.963838 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:22.963806 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a1ec59_ea70_461a_a9be_471315c4c7df.slice/crio-b28e65a575a5cc6a3761f676e6c7a0c05be11cc03a972b2db2827ce9cabc6461 WatchSource:0}: Error finding container b28e65a575a5cc6a3761f676e6c7a0c05be11cc03a972b2db2827ce9cabc6461: Status 404 returned error can't find the container with id b28e65a575a5cc6a3761f676e6c7a0c05be11cc03a972b2db2827ce9cabc6461 Jul 17 10:53:22.973332 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.973311 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fl9vh\" (UID: \"f41711b4-31e1-48c3-9f7f-1c9714d59b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:22.973466 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:22.973451 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 17 10:53:22.973517 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:22.973507 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls podName:f41711b4-31e1-48c3-9f7f-1c9714d59b06 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:30.973490861 +0000 UTC m=+74.647006491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fl9vh" (UID: "f41711b4-31e1-48c3-9f7f-1c9714d59b06") : secret "samples-operator-tls" not found Jul 17 10:53:22.980230 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:22.980209 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9p5ns"] Jul 17 10:53:22.982738 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:22.982718 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5041937f_62cc_401c_9c7e_3a15f4a6e88c.slice/crio-32abb6d537f28040cc3b7aa96b7332da9e55c7dce73a6029b8b8ce34cc3363e4 WatchSource:0}: Error finding container 32abb6d537f28040cc3b7aa96b7332da9e55c7dce73a6029b8b8ce34cc3363e4: Status 404 returned error can't find the container with id 32abb6d537f28040cc3b7aa96b7332da9e55c7dce73a6029b8b8ce34cc3363e4 Jul 17 10:53:23.108355 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:23.108269 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6m5q9" event={"ID":"91a1ec59-ea70-461a-a9be-471315c4c7df","Type":"ContainerStarted","Data":"b28e65a575a5cc6a3761f676e6c7a0c05be11cc03a972b2db2827ce9cabc6461"} Jul 17 10:53:23.109265 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:23.109241 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9p5ns" event={"ID":"5041937f-62cc-401c-9c7e-3a15f4a6e88c","Type":"ContainerStarted","Data":"32abb6d537f28040cc3b7aa96b7332da9e55c7dce73a6029b8b8ce34cc3363e4"} Jul 17 10:53:23.275822 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:23.275779 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:23.275979 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:23.275909 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 17 10:53:23.275979 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:23.275925 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b5c697c4d-rz846: secret "image-registry-tls" not found Jul 17 10:53:23.276104 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:23.275983 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls podName:4b8b6e43-da60-423a-9108-fda424fade1a nodeName:}" failed. No retries permitted until 2026-07-17 10:53:31.275964948 +0000 UTC m=+74.949480579 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls") pod "image-registry-6b5c697c4d-rz846" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a") : secret "image-registry-tls" not found Jul 17 10:53:23.578162 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:23.578130 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:23.578334 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:23.578220 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:23.578407 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:23.578370 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 10:53:23.578455 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:23.578437 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls podName:357ff6cd-3d67-44d5-9f4d-638229f579a6 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:27.578416796 +0000 UTC m=+71.251932446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2l2qk" (UID: "357ff6cd-3d67-44d5-9f4d-638229f579a6") : secret "insights-runtime-extractor-tls" not found Jul 17 10:53:23.578519 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:23.578507 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 10:53:23.578590 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:23.578567 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm podName:357ff6cd-3d67-44d5-9f4d-638229f579a6 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:27.578534861 +0000 UTC m=+71.252050509 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2l2qk" (UID: "357ff6cd-3d67-44d5-9f4d-638229f579a6") : configmap "kube-rbac-proxy" not found Jul 17 10:53:23.855355 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:23.855280 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lw2nf_38ad94cb-fac6-47f0-9024-9d1a71f16caf/serve-healthcheck-canary/0.log" Jul 17 10:53:24.587324 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:24.587274 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:53:24.587818 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:24.587450 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 17 10:53:24.587818 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:24.587512 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls podName:46c254d9-45eb-404e-a08e-60c8bbf2314e nodeName:}" failed. No retries permitted until 2026-07-17 10:53:56.587493344 +0000 UTC m=+100.261008990 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls") pod "dns-default-66wkj" (UID: "46c254d9-45eb-404e-a08e-60c8bbf2314e") : secret "dns-default-metrics-tls" not found Jul 17 10:53:25.805096 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:25.805054 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:25.805598 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:25.805131 2551 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:25.805598 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:25.805504 2551 scope.go:117] "RemoveContainer" containerID="eea949d5513ffa8edf33987ab0bda1629f8a036c6bfc420acf5f3d31891a4716" Jul 17 10:53:25.805710 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:25.805693 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-r6gsp_openshift-console-operator(55a709f2-51ec-4d79-923e-198fd980ba4b)\"" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" podUID="55a709f2-51ec-4d79-923e-198fd980ba4b" Jul 17 10:53:27.613923 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:27.613897 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:27.614192 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:27.613998 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:27.614192 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:27.614026 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 17 10:53:27.614192 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:27.614099 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm podName:357ff6cd-3d67-44d5-9f4d-638229f579a6 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:35.614077042 +0000 UTC m=+79.287592679 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2l2qk" (UID: "357ff6cd-3d67-44d5-9f4d-638229f579a6") : configmap "kube-rbac-proxy" not found Jul 17 10:53:27.614192 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:27.614162 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 17 10:53:27.614318 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:27.614221 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls podName:357ff6cd-3d67-44d5-9f4d-638229f579a6 nodeName:}" failed. No retries permitted until 2026-07-17 10:53:35.614206855 +0000 UTC m=+79.287722484 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2l2qk" (UID: "357ff6cd-3d67-44d5-9f4d-638229f579a6") : secret "insights-runtime-extractor-tls" not found Jul 17 10:53:28.125035 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:28.124990 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6m5q9" event={"ID":"91a1ec59-ea70-461a-a9be-471315c4c7df","Type":"ContainerStarted","Data":"f3c9365bd4e66dc1edad5ac66f57b7e803fdfe0fb45c8fc818b48b5a8d17c144"} Jul 17 10:53:28.126440 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:28.126414 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9p5ns" event={"ID":"5041937f-62cc-401c-9c7e-3a15f4a6e88c","Type":"ContainerStarted","Data":"10d6df1a62c5ceb8d80c32de30d69cc71ebb1e8e1f8bdad979424845f1468fb1"} Jul 17 10:53:28.126591 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:28.126542 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:53:28.141870 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:28.141825 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6m5q9" podStartSLOduration=66.557881954 podStartE2EDuration="1m11.141811595s" totalImagesPullingTime="4.583929641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:28.14059506 +0000 UTC m=+71.814110713" watchObservedRunningTime="2026-07-17 10:53:28.141811595 +0000 UTC m=+71.815327296" Jul 17 10:53:28.157450 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:28.157406 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9p5ns" podStartSLOduration=66.594814416 podStartE2EDuration="1m11.157395423s" totalImagesPullingTime="4.562581007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:28.156496779 +0000 UTC m=+71.830012434" watchObservedRunningTime="2026-07-17 10:53:28.157395423 +0000 UTC m=+71.830911074" Jul 17 10:53:31.043693 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:31.043652 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fl9vh\" (UID: \"f41711b4-31e1-48c3-9f7f-1c9714d59b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:31.045997 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:31.045977 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f41711b4-31e1-48c3-9f7f-1c9714d59b06-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fl9vh\" (UID: \"f41711b4-31e1-48c3-9f7f-1c9714d59b06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:31.167310 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:31.167276 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9dnn6\"" Jul 17 10:53:31.176167 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:31.176146 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" Jul 17 10:53:31.299361 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:31.299289 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh"] Jul 17 10:53:31.345176 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:31.345153 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:31.347243 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:31.347224 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls\") pod \"image-registry-6b5c697c4d-rz846\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:31.440709 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:31.440676 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gcsz5\"" Jul 17 10:53:31.448002 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:31.447974 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:31.573433 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:31.573311 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b5c697c4d-rz846"] Jul 17 10:53:31.575752 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:31.575727 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b8b6e43_da60_423a_9108_fda424fade1a.slice/crio-b16c9fa4f88fd29ba7e813eec9acb212bb92c1307b94537f3210f9ad2e105a1e WatchSource:0}: Error finding container b16c9fa4f88fd29ba7e813eec9acb212bb92c1307b94537f3210f9ad2e105a1e: Status 404 returned error can't find the container with id b16c9fa4f88fd29ba7e813eec9acb212bb92c1307b94537f3210f9ad2e105a1e Jul 17 10:53:32.139119 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:32.139088 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" event={"ID":"4b8b6e43-da60-423a-9108-fda424fade1a","Type":"ContainerStarted","Data":"e7690649eb57462a830932381041ce2375dbeb30ca51941dbd3eafdda799bedb"} Jul 17 10:53:32.139539 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:32.139138 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" event={"ID":"4b8b6e43-da60-423a-9108-fda424fade1a","Type":"ContainerStarted","Data":"b16c9fa4f88fd29ba7e813eec9acb212bb92c1307b94537f3210f9ad2e105a1e"} Jul 17 10:53:32.139539 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:32.139206 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:32.140311 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:32.140291 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" event={"ID":"f41711b4-31e1-48c3-9f7f-1c9714d59b06","Type":"ContainerStarted","Data":"ac1105badc68b0c528b188e2c171e62bd48707dacfe8eb9baefa24d3fd169039"} Jul 17 10:53:32.160357 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:32.160297 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" podStartSLOduration=17.160278354 podStartE2EDuration="17.160278354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:53:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:32.159043169 +0000 UTC m=+75.832558833" watchObservedRunningTime="2026-07-17 10:53:32.160278354 +0000 UTC m=+75.833794008" Jul 17 10:53:34.150702 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:34.150668 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" event={"ID":"f41711b4-31e1-48c3-9f7f-1c9714d59b06","Type":"ContainerStarted","Data":"888d30c38bf713970fcf0096ab947a428e21e1ea2d59c7802d7bc44ef9c3d649"} Jul 17 10:53:34.151085 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:34.150727 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" event={"ID":"f41711b4-31e1-48c3-9f7f-1c9714d59b06","Type":"ContainerStarted","Data":"933e0b21dfd9a0bd2fc383baf179e2b52d8569a140ef0859d491c8adf8a4b4f5"} Jul 17 10:53:34.172564 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:34.172505 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fl9vh" podStartSLOduration=17.403722585 podStartE2EDuration="19.172489031s" totalImagesPullingTime="1.768766446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:53:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:34.172161899 +0000 UTC m=+77.845677553" watchObservedRunningTime="2026-07-17 10:53:34.172489031 +0000 UTC m=+77.846004683" Jul 17 10:53:35.679879 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:35.679823 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:35.679879 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:35.679899 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:35.680413 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:35.680352 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/357ff6cd-3d67-44d5-9f4d-638229f579a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:35.682184 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:35.682158 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/357ff6cd-3d67-44d5-9f4d-638229f579a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2l2qk\" (UID: \"357ff6cd-3d67-44d5-9f4d-638229f579a6\") " pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:35.772595 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:35.772540 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2l2qk" Jul 17 10:53:35.914952 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:35.914919 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2l2qk"] Jul 17 10:53:35.918040 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:35.918014 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod357ff6cd_3d67_44d5_9f4d_638229f579a6.slice/crio-d5d92ec4c13a90da9232c7dacfcaa8dd522239b6c639ef4b7196b4f54ed0c33a WatchSource:0}: Error finding container d5d92ec4c13a90da9232c7dacfcaa8dd522239b6c639ef4b7196b4f54ed0c33a: Status 404 returned error can't find the container with id d5d92ec4c13a90da9232c7dacfcaa8dd522239b6c639ef4b7196b4f54ed0c33a Jul 17 10:53:36.157778 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:36.157742 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2l2qk" event={"ID":"357ff6cd-3d67-44d5-9f4d-638229f579a6","Type":"ContainerStarted","Data":"d5d92ec4c13a90da9232c7dacfcaa8dd522239b6c639ef4b7196b4f54ed0c33a"} Jul 17 10:53:36.819498 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:36.819467 2551 scope.go:117] "RemoveContainer" containerID="eea949d5513ffa8edf33987ab0bda1629f8a036c6bfc420acf5f3d31891a4716" Jul 17 10:53:37.163874 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:37.163790 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 10:53:37.164021 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:37.163884 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" event={"ID":"55a709f2-51ec-4d79-923e-198fd980ba4b","Type":"ContainerStarted","Data":"e772d724561de6231615f801a63bfeeb819714f5ff61bfc1287bf991b23922b6"} Jul 17 10:53:37.164734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:37.164347 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:37.190859 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:37.190810 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" podStartSLOduration=18.747221639 podStartE2EDuration="22.190795396s" totalImagesPullingTime="3.443573757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:53:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:37.190052915 +0000 UTC m=+80.863568570" watchObservedRunningTime="2026-07-17 10:53:37.190795396 +0000 UTC m=+80.864311064" Jul 17 10:53:37.909666 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:37.909629 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-r6gsp" Jul 17 10:53:41.178349 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:41.178267 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2l2qk" event={"ID":"357ff6cd-3d67-44d5-9f4d-638229f579a6","Type":"ContainerStarted","Data":"5e18393dd2c8c7fdf6168557ddd4a6757328328736b31d42bb08ffd554cfee65"} Jul 17 10:53:41.981160 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:41.981128 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc"] Jul 17 10:53:41.984314 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:41.984285 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:41.989151 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:41.989125 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 17 10:53:41.989151 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:41.989135 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 17 10:53:41.990175 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:41.990152 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 17 10:53:41.991719 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:41.991696 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 17 10:53:41.998835 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:41.998814 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc"] Jul 17 10:53:42.034115 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.034085 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9"] Jul 17 10:53:42.037230 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.037205 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9" Jul 17 10:53:42.047567 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.047529 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-s2pcl\"" Jul 17 10:53:42.047690 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.047662 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 17 10:53:42.054635 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.054613 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9"] Jul 17 10:53:42.100204 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.100165 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b5c697c4d-rz846"] Jul 17 10:53:42.136530 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.136492 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-488qb\" (UniqueName: \"kubernetes.io/projected/cbefd52c-40c2-4656-9d3b-77d5f54f03e2-kube-api-access-488qb\") pod \"klusterlet-addon-workmgr-5c4d786797-8thpc\" (UID: \"cbefd52c-40c2-4656-9d3b-77d5f54f03e2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:42.136716 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.136540 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbefd52c-40c2-4656-9d3b-77d5f54f03e2-tmp\") pod \"klusterlet-addon-workmgr-5c4d786797-8thpc\" (UID: \"cbefd52c-40c2-4656-9d3b-77d5f54f03e2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:42.136716 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.136643 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/cbefd52c-40c2-4656-9d3b-77d5f54f03e2-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c4d786797-8thpc\" (UID: \"cbefd52c-40c2-4656-9d3b-77d5f54f03e2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:42.136839 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.136731 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3345c573-10d3-433e-b9ee-42a5bfd6c3a1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wxgr9\" (UID: \"3345c573-10d3-433e-b9ee-42a5bfd6c3a1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9" Jul 17 10:53:42.237430 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.237342 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-488qb\" (UniqueName: \"kubernetes.io/projected/cbefd52c-40c2-4656-9d3b-77d5f54f03e2-kube-api-access-488qb\") pod \"klusterlet-addon-workmgr-5c4d786797-8thpc\" (UID: \"cbefd52c-40c2-4656-9d3b-77d5f54f03e2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:42.237430 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.237397 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbefd52c-40c2-4656-9d3b-77d5f54f03e2-tmp\") pod \"klusterlet-addon-workmgr-5c4d786797-8thpc\" (UID: \"cbefd52c-40c2-4656-9d3b-77d5f54f03e2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:42.237974 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.237444 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/cbefd52c-40c2-4656-9d3b-77d5f54f03e2-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c4d786797-8thpc\" (UID: \"cbefd52c-40c2-4656-9d3b-77d5f54f03e2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:42.237974 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.237696 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3345c573-10d3-433e-b9ee-42a5bfd6c3a1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wxgr9\" (UID: \"3345c573-10d3-433e-b9ee-42a5bfd6c3a1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9" Jul 17 10:53:42.237974 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.237883 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbefd52c-40c2-4656-9d3b-77d5f54f03e2-tmp\") pod \"klusterlet-addon-workmgr-5c4d786797-8thpc\" (UID: \"cbefd52c-40c2-4656-9d3b-77d5f54f03e2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:42.240364 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.240330 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/cbefd52c-40c2-4656-9d3b-77d5f54f03e2-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c4d786797-8thpc\" (UID: \"cbefd52c-40c2-4656-9d3b-77d5f54f03e2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:42.240364 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.240332 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3345c573-10d3-433e-b9ee-42a5bfd6c3a1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wxgr9\" (UID: \"3345c573-10d3-433e-b9ee-42a5bfd6c3a1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9" Jul 17 10:53:42.271045 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.271016 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-488qb\" (UniqueName: \"kubernetes.io/projected/cbefd52c-40c2-4656-9d3b-77d5f54f03e2-kube-api-access-488qb\") pod \"klusterlet-addon-workmgr-5c4d786797-8thpc\" (UID: \"cbefd52c-40c2-4656-9d3b-77d5f54f03e2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:42.296829 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.296795 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:42.348039 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.347990 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9" Jul 17 10:53:42.447393 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.447349 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc"] Jul 17 10:53:42.496093 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:42.496021 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9"] Jul 17 10:53:43.683686 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:43.683651 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbefd52c_40c2_4656_9d3b_77d5f54f03e2.slice/crio-db0a36f6b8a98110d0d7399e7ba5fb1df2c6bcd5e79f492c03b1ab661ba943a0 WatchSource:0}: Error finding container db0a36f6b8a98110d0d7399e7ba5fb1df2c6bcd5e79f492c03b1ab661ba943a0: Status 404 returned error can't find the container with id db0a36f6b8a98110d0d7399e7ba5fb1df2c6bcd5e79f492c03b1ab661ba943a0 Jul 17 10:53:43.684449 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:43.684420 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3345c573_10d3_433e_b9ee_42a5bfd6c3a1.slice/crio-4c9d7c9e5d99ae4a0f4c9e19e007781e0949f6fa06aed2b1c2fcd5ac2248024e WatchSource:0}: Error finding container 4c9d7c9e5d99ae4a0f4c9e19e007781e0949f6fa06aed2b1c2fcd5ac2248024e: Status 404 returned error can't find the container with id 4c9d7c9e5d99ae4a0f4c9e19e007781e0949f6fa06aed2b1c2fcd5ac2248024e Jul 17 10:53:44.190401 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:44.190352 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9" event={"ID":"3345c573-10d3-433e-b9ee-42a5bfd6c3a1","Type":"ContainerStarted","Data":"4c9d7c9e5d99ae4a0f4c9e19e007781e0949f6fa06aed2b1c2fcd5ac2248024e"} Jul 17 10:53:44.191609 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:44.191579 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" event={"ID":"cbefd52c-40c2-4656-9d3b-77d5f54f03e2","Type":"ContainerStarted","Data":"db0a36f6b8a98110d0d7399e7ba5fb1df2c6bcd5e79f492c03b1ab661ba943a0"} Jul 17 10:53:45.198304 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:45.198210 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2l2qk" event={"ID":"357ff6cd-3d67-44d5-9f4d-638229f579a6","Type":"ContainerStarted","Data":"7c214f237e3a49f73969b0d85ff3005d54e8709c213dbd9f5f8723b3b47a2819"} Jul 17 10:53:46.205522 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:46.205429 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9" event={"ID":"3345c573-10d3-433e-b9ee-42a5bfd6c3a1","Type":"ContainerStarted","Data":"e7496309708a63d233bd8ecaa28b2872be1c9a907ab911326aae6d6f24286468"} Jul 17 10:53:46.206112 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:46.206047 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9" Jul 17 10:53:46.212467 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:46.212420 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9" Jul 17 10:53:46.223259 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:46.223105 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wxgr9" podStartSLOduration=3.678335311 podStartE2EDuration="5.223086327s" totalImagesPullingTime="1.544751016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:53:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:46.223010462 +0000 UTC m=+89.896526115" watchObservedRunningTime="2026-07-17 10:53:46.223086327 +0000 UTC m=+89.896601979" Jul 17 10:53:47.061941 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.061906 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-2m52h"] Jul 17 10:53:47.065203 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.065176 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.068210 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.068189 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 17 10:53:47.069005 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.068982 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 17 10:53:47.069341 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.069320 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 17 10:53:47.069746 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.069727 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 17 10:53:47.069842 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.069732 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tgk8p\"" Jul 17 10:53:47.072929 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.072912 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 17 10:53:47.084490 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.084459 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-2m52h"] Jul 17 10:53:47.180237 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.180206 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f6b14f2-08d2-41a4-be04-511ac0c901ee-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.180408 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.180343 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f6b14f2-08d2-41a4-be04-511ac0c901ee-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.180408 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.180383 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm2nf\" (UniqueName: \"kubernetes.io/projected/7f6b14f2-08d2-41a4-be04-511ac0c901ee-kube-api-access-vm2nf\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.180524 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.180492 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f6b14f2-08d2-41a4-be04-511ac0c901ee-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.281256 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.281213 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f6b14f2-08d2-41a4-be04-511ac0c901ee-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.281789 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.281265 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm2nf\" (UniqueName: \"kubernetes.io/projected/7f6b14f2-08d2-41a4-be04-511ac0c901ee-kube-api-access-vm2nf\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.281789 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.281325 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f6b14f2-08d2-41a4-be04-511ac0c901ee-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.281789 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.281365 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f6b14f2-08d2-41a4-be04-511ac0c901ee-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.281789 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:47.281507 2551 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 17 10:53:47.281789 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:47.281596 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6b14f2-08d2-41a4-be04-511ac0c901ee-prometheus-operator-tls podName:7f6b14f2-08d2-41a4-be04-511ac0c901ee nodeName:}" failed. No retries permitted until 2026-07-17 10:53:47.781574368 +0000 UTC m=+91.455090008 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7f6b14f2-08d2-41a4-be04-511ac0c901ee-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-2m52h" (UID: "7f6b14f2-08d2-41a4-be04-511ac0c901ee") : secret "prometheus-operator-tls" not found Jul 17 10:53:47.282124 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.282100 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f6b14f2-08d2-41a4-be04-511ac0c901ee-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.284077 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.284058 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f6b14f2-08d2-41a4-be04-511ac0c901ee-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.295910 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.295886 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm2nf\" (UniqueName: \"kubernetes.io/projected/7f6b14f2-08d2-41a4-be04-511ac0c901ee-kube-api-access-vm2nf\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.786723 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.786682 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f6b14f2-08d2-41a4-be04-511ac0c901ee-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.789323 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.789291 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f6b14f2-08d2-41a4-be04-511ac0c901ee-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-2m52h\" (UID: \"7f6b14f2-08d2-41a4-be04-511ac0c901ee\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:47.975621 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:47.975580 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" Jul 17 10:53:48.429297 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.429265 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b4b7f87d6-p5k48"] Jul 17 10:53:48.443291 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.443264 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.447010 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.446831 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 17 10:53:48.447010 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.446901 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 17 10:53:48.447595 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.447543 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ck8c2\"" Jul 17 10:53:48.447723 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.447633 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 17 10:53:48.447723 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.447649 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b4b7f87d6-p5k48"] Jul 17 10:53:48.447723 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.447580 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 17 10:53:48.449661 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.449292 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 17 10:53:48.449661 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.449540 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 17 10:53:48.450053 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.450005 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 17 10:53:48.594323 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.594286 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx2t5\" (UniqueName: \"kubernetes.io/projected/962b5236-4918-487b-a657-fd99d9224722-kube-api-access-tx2t5\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.594585 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.594455 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-console-config\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.594585 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.594496 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-oauth-config\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.594585 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.594574 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-serving-cert\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.594761 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.594614 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-oauth-serving-cert\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.594761 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.594650 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-service-ca\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.695428 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.695348 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-serving-cert\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.695428 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.695393 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-oauth-serving-cert\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.695667 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.695516 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-service-ca\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.695667 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.695596 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx2t5\" (UniqueName: \"kubernetes.io/projected/962b5236-4918-487b-a657-fd99d9224722-kube-api-access-tx2t5\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.695775 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.695699 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-console-config\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.695775 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.695729 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-oauth-config\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.696177 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.696141 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-service-ca\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.696177 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.696141 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-oauth-serving-cert\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.696395 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.696372 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-console-config\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.698078 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.698054 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-serving-cert\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.698201 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.698105 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-oauth-config\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.704865 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.704840 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx2t5\" (UniqueName: \"kubernetes.io/projected/962b5236-4918-487b-a657-fd99d9224722-kube-api-access-tx2t5\") pod \"console-6b4b7f87d6-p5k48\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:48.754608 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:48.754571 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:49.485780 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:49.485747 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-2m52h"] Jul 17 10:53:49.489710 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:49.489675 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6b14f2_08d2_41a4_be04_511ac0c901ee.slice/crio-1ff725ecdd72e7c09242300caf3b4420e18c3a3a858f791d2bde06891b17c818 WatchSource:0}: Error finding container 1ff725ecdd72e7c09242300caf3b4420e18c3a3a858f791d2bde06891b17c818: Status 404 returned error can't find the container with id 1ff725ecdd72e7c09242300caf3b4420e18c3a3a858f791d2bde06891b17c818 Jul 17 10:53:49.492528 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:49.492503 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b4b7f87d6-p5k48"] Jul 17 10:53:49.495648 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:49.495626 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962b5236_4918_487b_a657_fd99d9224722.slice/crio-b49bc36ccb45691844e517b1c66b69d7063f33bfe55a67928c2dc318d953a289 WatchSource:0}: Error finding container b49bc36ccb45691844e517b1c66b69d7063f33bfe55a67928c2dc318d953a289: Status 404 returned error can't find the container with id b49bc36ccb45691844e517b1c66b69d7063f33bfe55a67928c2dc318d953a289 Jul 17 10:53:50.226074 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:50.226034 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" event={"ID":"7f6b14f2-08d2-41a4-be04-511ac0c901ee","Type":"ContainerStarted","Data":"1ff725ecdd72e7c09242300caf3b4420e18c3a3a858f791d2bde06891b17c818"} Jul 17 10:53:50.228505 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:50.228476 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2l2qk" event={"ID":"357ff6cd-3d67-44d5-9f4d-638229f579a6","Type":"ContainerStarted","Data":"e1b4aacd18f6feb634b10d35308e1ba3191129d463b22902ac1ad771c1e0d87c"} Jul 17 10:53:50.230289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:50.230255 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" event={"ID":"cbefd52c-40c2-4656-9d3b-77d5f54f03e2","Type":"ContainerStarted","Data":"1f044a47d596522f4bb15c9863872ef08179bb47517800f07fb3bab1dd6dbc69"} Jul 17 10:53:50.230478 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:50.230460 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:50.231620 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:50.231593 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4b7f87d6-p5k48" event={"ID":"962b5236-4918-487b-a657-fd99d9224722","Type":"ContainerStarted","Data":"b49bc36ccb45691844e517b1c66b69d7063f33bfe55a67928c2dc318d953a289"} Jul 17 10:53:50.232680 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:50.232655 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" Jul 17 10:53:50.252263 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:50.252209 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2l2qk" podStartSLOduration=17.964607467 podStartE2EDuration="31.252193547s" totalImagesPullingTime="13.28758608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:53:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:50.251085023 +0000 UTC m=+93.924600674" watchObservedRunningTime="2026-07-17 10:53:50.252193547 +0000 UTC m=+93.925709198" Jul 17 10:53:50.269904 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:50.269823 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c4d786797-8thpc" podStartSLOduration=3.624800905 podStartE2EDuration="9.269805137s" totalImagesPullingTime="5.645004232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:53:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:50.268274741 +0000 UTC m=+93.941790404" watchObservedRunningTime="2026-07-17 10:53:50.269805137 +0000 UTC m=+93.943320803" Jul 17 10:53:51.237856 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:51.237756 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" event={"ID":"7f6b14f2-08d2-41a4-be04-511ac0c901ee","Type":"ContainerStarted","Data":"aac96aaabcafeba7113f6e63c4c48cf190c9e04bb7b46fa8d0fdd67338cabb73"} Jul 17 10:53:51.237856 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:51.237821 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" event={"ID":"7f6b14f2-08d2-41a4-be04-511ac0c901ee","Type":"ContainerStarted","Data":"f997e6634257beeb4b5b0072d31e90d70a6cd8283eff3bf1e5441870825f6840"} Jul 17 10:53:51.258166 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:51.258110 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-2m52h" podStartSLOduration=2.955456199 podStartE2EDuration="4.25809434s" totalImagesPullingTime="1.302638141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:51.256684508 +0000 UTC m=+94.930200154" watchObservedRunningTime="2026-07-17 10:53:51.25809434 +0000 UTC m=+94.931609991" Jul 17 10:53:52.106247 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:52.106206 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:53:53.246816 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.246785 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4b7f87d6-p5k48" event={"ID":"962b5236-4918-487b-a657-fd99d9224722","Type":"ContainerStarted","Data":"73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad"} Jul 17 10:53:53.263043 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.262992 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b4b7f87d6-p5k48" podStartSLOduration=2.105154813 podStartE2EDuration="5.262977614s" totalImagesPullingTime="3.157822801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:53:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:53.26221622 +0000 UTC m=+96.935731871" watchObservedRunningTime="2026-07-17 10:53:53.262977614 +0000 UTC m=+96.936493244" Jul 17 10:53:53.450966 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.450935 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2fdjf"] Jul 17 10:53:53.454131 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.454116 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.460779 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.460753 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 17 10:53:53.460909 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.460756 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 17 10:53:53.463757 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.463738 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 17 10:53:53.469737 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.469720 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sfnqc\"" Jul 17 10:53:53.644397 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.644362 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05445548-c931-4b7c-ac99-c8562fa1aa23-sys\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.644397 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.644398 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-tls\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.644643 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.644429 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-wtmp\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.644643 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.644450 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.644643 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.644480 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05445548-c931-4b7c-ac99-c8562fa1aa23-metrics-client-ca\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.644643 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.644496 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-textfile\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.644643 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.644513 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-accelerators-collector-config\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.644643 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.644530 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sm2x\" (UniqueName: \"kubernetes.io/projected/05445548-c931-4b7c-ac99-c8562fa1aa23-kube-api-access-2sm2x\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.644643 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.644632 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/05445548-c931-4b7c-ac99-c8562fa1aa23-root\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.745556 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.745518 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.745732 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.745590 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05445548-c931-4b7c-ac99-c8562fa1aa23-metrics-client-ca\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.745732 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.745612 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-textfile\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.745732 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.745633 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-accelerators-collector-config\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.745732 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.745654 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sm2x\" (UniqueName: \"kubernetes.io/projected/05445548-c931-4b7c-ac99-c8562fa1aa23-kube-api-access-2sm2x\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.745907 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.745736 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/05445548-c931-4b7c-ac99-c8562fa1aa23-root\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.745907 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.745793 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05445548-c931-4b7c-ac99-c8562fa1aa23-sys\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.745907 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.745820 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-tls\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.745907 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.745878 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-wtmp\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.745907 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.745892 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/05445548-c931-4b7c-ac99-c8562fa1aa23-root\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.746138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.745900 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05445548-c931-4b7c-ac99-c8562fa1aa23-sys\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.746138 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.746114 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-wtmp\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.746316 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.746290 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-textfile\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.746421 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.746314 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-accelerators-collector-config\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.746421 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.746403 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05445548-c931-4b7c-ac99-c8562fa1aa23-metrics-client-ca\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.748098 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.748073 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.748183 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.748073 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/05445548-c931-4b7c-ac99-c8562fa1aa23-node-exporter-tls\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.755077 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.755054 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sm2x\" (UniqueName: \"kubernetes.io/projected/05445548-c931-4b7c-ac99-c8562fa1aa23-kube-api-access-2sm2x\") pod \"node-exporter-2fdjf\" (UID: \"05445548-c931-4b7c-ac99-c8562fa1aa23\") " pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.762850 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:53.762831 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2fdjf" Jul 17 10:53:53.775127 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:53.775106 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05445548_c931_4b7c_ac99_c8562fa1aa23.slice/crio-909aead44a6631d134e0a2bd9da8b90b966ab340244a9a7c2142a9bb9a98f4de WatchSource:0}: Error finding container 909aead44a6631d134e0a2bd9da8b90b966ab340244a9a7c2142a9bb9a98f4de: Status 404 returned error can't find the container with id 909aead44a6631d134e0a2bd9da8b90b966ab340244a9a7c2142a9bb9a98f4de Jul 17 10:53:54.251465 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:54.251423 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fdjf" event={"ID":"05445548-c931-4b7c-ac99-c8562fa1aa23","Type":"ContainerStarted","Data":"909aead44a6631d134e0a2bd9da8b90b966ab340244a9a7c2142a9bb9a98f4de"} Jul 17 10:53:55.257916 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:55.257885 2551 generic.go:356] "Generic (PLEG): container finished" podID="05445548-c931-4b7c-ac99-c8562fa1aa23" containerID="16bd4c8076aa3ce58a8b1c71bc4f83c8f1a7f5657c2652e26eb47274b0a535c2" exitCode=0 Jul 17 10:53:55.258293 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:55.257923 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fdjf" event={"ID":"05445548-c931-4b7c-ac99-c8562fa1aa23","Type":"ContainerDied","Data":"16bd4c8076aa3ce58a8b1c71bc4f83c8f1a7f5657c2652e26eb47274b0a535c2"} Jul 17 10:53:56.264317 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.264281 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fdjf" event={"ID":"05445548-c931-4b7c-ac99-c8562fa1aa23","Type":"ContainerStarted","Data":"839fd4bbc3bcf5cbc481574e1b15e53fb032c678fde5cef9268104243d66e64b"} Jul 17 10:53:56.264711 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.264334 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fdjf" event={"ID":"05445548-c931-4b7c-ac99-c8562fa1aa23","Type":"ContainerStarted","Data":"7a7c62c1a1cdcc6d13d289b1b626efd0a11413f51130cd8a80e79aa437bf264d"} Jul 17 10:53:56.288295 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.288244 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2fdjf" podStartSLOduration=2.182481704 podStartE2EDuration="3.288228277s" totalImagesPullingTime="1.044038281s" totalInitContainerRuntime="61.708292ms" isStatefulPod=true podCreationTimestamp="2026-07-17 10:53:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:56.287062097 +0000 UTC m=+99.960577750" watchObservedRunningTime="2026-07-17 10:53:56.288228277 +0000 UTC m=+99.961743929" Jul 17 10:53:56.503295 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.503257 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78c87d5d66-86gsj"] Jul 17 10:53:56.507053 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.507034 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.509942 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.509920 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 17 10:53:56.510082 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.509929 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 17 10:53:56.510216 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.510200 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e7s0j1h4md2t4\"" Jul 17 10:53:56.510258 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.510223 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 17 10:53:56.510302 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.510256 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-btpwf\"" Jul 17 10:53:56.510353 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.510222 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 17 10:53:56.510471 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.510456 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 17 10:53:56.531245 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.531223 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78c87d5d66-86gsj"] Jul 17 10:53:56.564179 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.564144 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.564349 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.564189 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-tls\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.564349 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.564217 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.564349 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.564259 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f5k6\" (UniqueName: \"kubernetes.io/projected/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-kube-api-access-9f5k6\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.564472 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.564385 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.564472 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.564411 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-grpc-tls\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.564472 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.564432 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-metrics-client-ca\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.564472 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.564451 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.665879 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.665843 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:53:56.666059 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.665892 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.666059 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.665927 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-grpc-tls\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.666059 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.666002 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-metrics-client-ca\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.666059 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.666051 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.666227 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.666123 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.666317 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.666274 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-tls\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.666393 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.666348 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.666393 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.666379 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f5k6\" (UniqueName: \"kubernetes.io/projected/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-kube-api-access-9f5k6\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.666928 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.666785 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-metrics-client-ca\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.668934 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.668906 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.669038 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.668983 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46c254d9-45eb-404e-a08e-60c8bbf2314e-metrics-tls\") pod \"dns-default-66wkj\" (UID: \"46c254d9-45eb-404e-a08e-60c8bbf2314e\") " pod="openshift-dns/dns-default-66wkj" Jul 17 10:53:56.669038 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.669004 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.669106 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.669057 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.669106 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.669069 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-grpc-tls\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.669323 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.669307 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.669409 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.669388 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-secret-thanos-querier-tls\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.674629 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.674611 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f5k6\" (UniqueName: \"kubernetes.io/projected/9ddf30d1-4651-443a-9ae8-d2bd0d8914a4-kube-api-access-9f5k6\") pod \"thanos-querier-78c87d5d66-86gsj\" (UID: \"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4\") " pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.816487 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.816393 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:53:56.920542 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.920512 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dnhpd\"" Jul 17 10:53:56.924028 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.924003 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-66wkj" Jul 17 10:53:56.958168 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:56.958134 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78c87d5d66-86gsj"] Jul 17 10:53:56.961292 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:56.961262 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ddf30d1_4651_443a_9ae8_d2bd0d8914a4.slice/crio-a12c59a1852765a29064d9078bd1c0348391c943936375a26af12ec9207fc4c2 WatchSource:0}: Error finding container a12c59a1852765a29064d9078bd1c0348391c943936375a26af12ec9207fc4c2: Status 404 returned error can't find the container with id a12c59a1852765a29064d9078bd1c0348391c943936375a26af12ec9207fc4c2 Jul 17 10:53:57.019042 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.019010 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7445f6d84-j452c"] Jul 17 10:53:57.023717 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.023688 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.032414 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.032378 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 17 10:53:57.035979 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.035934 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7445f6d84-j452c"] Jul 17 10:53:57.058502 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.058472 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-66wkj"] Jul 17 10:53:57.062591 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:57.062540 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c254d9_45eb_404e_a08e_60c8bbf2314e.slice/crio-953c3e352d39b9f949cefecf816e1a05d5c261b168f67c872e0b4bc04f57cee5 WatchSource:0}: Error finding container 953c3e352d39b9f949cefecf816e1a05d5c261b168f67c872e0b4bc04f57cee5: Status 404 returned error can't find the container with id 953c3e352d39b9f949cefecf816e1a05d5c261b168f67c872e0b4bc04f57cee5 Jul 17 10:53:57.070368 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.070318 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-config\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.070368 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.070352 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-trusted-ca-bundle\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.070483 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.070370 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64f58\" (UniqueName: \"kubernetes.io/projected/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-kube-api-access-64f58\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.070483 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.070470 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-service-ca\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.070567 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.070498 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-serving-cert\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.070567 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.070520 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-oauth-config\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.070684 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.070671 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-oauth-serving-cert\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.171750 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.171718 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-trusted-ca-bundle\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.171750 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.171756 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64f58\" (UniqueName: \"kubernetes.io/projected/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-kube-api-access-64f58\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.171945 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.171839 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-service-ca\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.171945 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.171892 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-serving-cert\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.171945 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.171926 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-oauth-config\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.172081 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.171964 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-oauth-serving-cert\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.172081 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.171999 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-config\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.172654 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.172624 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-service-ca\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.172799 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.172651 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-oauth-serving-cert\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.172799 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.172724 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-config\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.172799 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.172753 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-trusted-ca-bundle\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.174357 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.174327 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-oauth-config\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.174467 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.174449 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-serving-cert\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.180994 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.180978 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64f58\" (UniqueName: \"kubernetes.io/projected/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-kube-api-access-64f58\") pod \"console-7445f6d84-j452c\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.269091 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.269053 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-66wkj" event={"ID":"46c254d9-45eb-404e-a08e-60c8bbf2314e","Type":"ContainerStarted","Data":"953c3e352d39b9f949cefecf816e1a05d5c261b168f67c872e0b4bc04f57cee5"} Jul 17 10:53:57.270229 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.270200 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" event={"ID":"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4","Type":"ContainerStarted","Data":"a12c59a1852765a29064d9078bd1c0348391c943936375a26af12ec9207fc4c2"} Jul 17 10:53:57.333990 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.333904 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:53:57.477264 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:57.477094 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7445f6d84-j452c"] Jul 17 10:53:57.479950 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:57.479920 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a71433_3be4_4dfe_b5c4_0bf2ce4896aa.slice/crio-c8202482f405d1f6a9f76143c933001a7ff706330876b805bd402f038f58373b WatchSource:0}: Error finding container c8202482f405d1f6a9f76143c933001a7ff706330876b805bd402f038f58373b: Status 404 returned error can't find the container with id c8202482f405d1f6a9f76143c933001a7ff706330876b805bd402f038f58373b Jul 17 10:53:58.186916 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.186873 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2"] Jul 17 10:53:58.190390 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.190365 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" Jul 17 10:53:58.193474 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.193107 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 17 10:53:58.193474 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.193360 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dcv4f\"" Jul 17 10:53:58.202399 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.202339 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2"] Jul 17 10:53:58.277852 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.277598 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7445f6d84-j452c" event={"ID":"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa","Type":"ContainerStarted","Data":"ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985"} Jul 17 10:53:58.277852 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.277662 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7445f6d84-j452c" event={"ID":"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa","Type":"ContainerStarted","Data":"c8202482f405d1f6a9f76143c933001a7ff706330876b805bd402f038f58373b"} Jul 17 10:53:58.281015 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.280983 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e34bda92-1aef-4cf9-9535-f3fdb9ed14ef-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-pctt2\" (UID: \"e34bda92-1aef-4cf9-9535-f3fdb9ed14ef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" Jul 17 10:53:58.281151 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.281040 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cj8h\" (UniqueName: \"kubernetes.io/projected/e34bda92-1aef-4cf9-9535-f3fdb9ed14ef-kube-api-access-2cj8h\") pod \"monitoring-plugin-7b7b7bc8bf-pctt2\" (UID: \"e34bda92-1aef-4cf9-9535-f3fdb9ed14ef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" Jul 17 10:53:58.298055 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.298003 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7445f6d84-j452c" podStartSLOduration=2.29798071 podStartE2EDuration="2.29798071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:53:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:58.297412064 +0000 UTC m=+101.970927716" watchObservedRunningTime="2026-07-17 10:53:58.29798071 +0000 UTC m=+101.971496459" Jul 17 10:53:58.382055 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.382011 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e34bda92-1aef-4cf9-9535-f3fdb9ed14ef-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-pctt2\" (UID: \"e34bda92-1aef-4cf9-9535-f3fdb9ed14ef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" Jul 17 10:53:58.382240 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.382086 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cj8h\" (UniqueName: \"kubernetes.io/projected/e34bda92-1aef-4cf9-9535-f3fdb9ed14ef-kube-api-access-2cj8h\") pod \"monitoring-plugin-7b7b7bc8bf-pctt2\" (UID: \"e34bda92-1aef-4cf9-9535-f3fdb9ed14ef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" Jul 17 10:53:58.382240 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:58.382159 2551 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 17 10:53:58.382240 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:53:58.382234 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e34bda92-1aef-4cf9-9535-f3fdb9ed14ef-monitoring-plugin-cert podName:e34bda92-1aef-4cf9-9535-f3fdb9ed14ef nodeName:}" failed. No retries permitted until 2026-07-17 10:53:58.882210681 +0000 UTC m=+102.555726332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e34bda92-1aef-4cf9-9535-f3fdb9ed14ef-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-pctt2" (UID: "e34bda92-1aef-4cf9-9535-f3fdb9ed14ef") : secret "monitoring-plugin-cert" not found Jul 17 10:53:58.391994 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.391962 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cj8h\" (UniqueName: \"kubernetes.io/projected/e34bda92-1aef-4cf9-9535-f3fdb9ed14ef-kube-api-access-2cj8h\") pod \"monitoring-plugin-7b7b7bc8bf-pctt2\" (UID: \"e34bda92-1aef-4cf9-9535-f3fdb9ed14ef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" Jul 17 10:53:58.755530 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.755503 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:58.755674 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.755587 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:58.760777 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.760748 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:58.885412 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.885325 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e34bda92-1aef-4cf9-9535-f3fdb9ed14ef-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-pctt2\" (UID: \"e34bda92-1aef-4cf9-9535-f3fdb9ed14ef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" Jul 17 10:53:58.888135 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:58.888105 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e34bda92-1aef-4cf9-9535-f3fdb9ed14ef-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-pctt2\" (UID: \"e34bda92-1aef-4cf9-9535-f3fdb9ed14ef\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" Jul 17 10:53:59.105674 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:59.105633 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" Jul 17 10:53:59.132203 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:59.132173 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9p5ns" Jul 17 10:53:59.285438 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:59.285400 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-66wkj" event={"ID":"46c254d9-45eb-404e-a08e-60c8bbf2314e","Type":"ContainerStarted","Data":"f4f39f1be3820ae86e4b730c92afa6cfe57323a18aae157afb363a6ab16a617f"} Jul 17 10:53:59.285917 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:59.285469 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-66wkj" event={"ID":"46c254d9-45eb-404e-a08e-60c8bbf2314e","Type":"ContainerStarted","Data":"5ff30b00ac16dcc70928ac9535343a22572fe9044aa4e117e3f77f9ee93f4edb"} Jul 17 10:53:59.290830 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:59.290795 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:53:59.307056 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:59.307004 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-66wkj" podStartSLOduration=65.77985706 podStartE2EDuration="1m7.306986578s" totalImagesPullingTime="1.527129518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:53:59.305287713 +0000 UTC m=+102.978803365" watchObservedRunningTime="2026-07-17 10:53:59.306986578 +0000 UTC m=+102.980502229" Jul 17 10:53:59.462889 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:53:59.462849 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2"] Jul 17 10:53:59.466667 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:53:59.466640 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode34bda92_1aef_4cf9_9535_f3fdb9ed14ef.slice/crio-8de0aeaf0adafdae12afd8344b421a27ef3c72fbca9f1af97a302860a478cee7 WatchSource:0}: Error finding container 8de0aeaf0adafdae12afd8344b421a27ef3c72fbca9f1af97a302860a478cee7: Status 404 returned error can't find the container with id 8de0aeaf0adafdae12afd8344b421a27ef3c72fbca9f1af97a302860a478cee7 Jul 17 10:54:00.291531 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:00.291494 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" event={"ID":"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4","Type":"ContainerStarted","Data":"40fdcde0d37dce8b8946da9f97ec9105aea309c3eeab30dea4f2f82c0cbf22f1"} Jul 17 10:54:00.291955 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:00.291594 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" event={"ID":"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4","Type":"ContainerStarted","Data":"0188449b43ad9feaf555545b050cbd55934fe821c7433fb28c605a4799e28159"} Jul 17 10:54:00.291955 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:00.291640 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" event={"ID":"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4","Type":"ContainerStarted","Data":"de1f611bf155ef7090a5a52599111d37f15fc4489c843e765757b97c662e43a7"} Jul 17 10:54:00.292735 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:00.292713 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" event={"ID":"e34bda92-1aef-4cf9-9535-f3fdb9ed14ef","Type":"ContainerStarted","Data":"8de0aeaf0adafdae12afd8344b421a27ef3c72fbca9f1af97a302860a478cee7"} Jul 17 10:54:00.293066 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:00.293054 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-66wkj" Jul 17 10:54:01.298639 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:01.298600 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" event={"ID":"e34bda92-1aef-4cf9-9535-f3fdb9ed14ef","Type":"ContainerStarted","Data":"3e23e9d1c5257303175f68940643a49c8b92db8228fce824bcfe72c4c39a38ec"} Jul 17 10:54:01.299102 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:01.298978 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" Jul 17 10:54:01.304417 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:01.304392 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" Jul 17 10:54:01.318135 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:01.318081 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pctt2" podStartSLOduration=1.663888298 podStartE2EDuration="3.318065643s" totalImagesPullingTime="1.654177345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:53:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:54:01.317051315 +0000 UTC m=+104.990566968" watchObservedRunningTime="2026-07-17 10:54:01.318065643 +0000 UTC m=+104.991581298" Jul 17 10:54:02.305150 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:02.305113 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" event={"ID":"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4","Type":"ContainerStarted","Data":"b1141de1bfd4c5e147acb358c75e221ff5b8a104819bdbc599dd224e18ddb7c5"} Jul 17 10:54:02.305515 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:02.305205 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" event={"ID":"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4","Type":"ContainerStarted","Data":"978b0826f1c2ed2ae3e3c4a3e1bb4ef1fcbfbc06886686514369b5c3d7e64c3b"} Jul 17 10:54:02.305515 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:02.305237 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" event={"ID":"9ddf30d1-4651-443a-9ae8-d2bd0d8914a4","Type":"ContainerStarted","Data":"203f2488302c2aa55086118aa3e06a18ddb83c3430d5a1e5b8eb9760b529e25d"} Jul 17 10:54:02.335957 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:02.335905 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" podStartSLOduration=1.6957873650000002 podStartE2EDuration="6.33588771s" totalImagesPullingTime="4.640100345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:53:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:54:02.333701797 +0000 UTC m=+106.007217460" watchObservedRunningTime="2026-07-17 10:54:02.33588771 +0000 UTC m=+106.009403401" Jul 17 10:54:03.309817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:03.309788 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:54:04.319116 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.319086 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78c87d5d66-86gsj" Jul 17 10:54:04.493202 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.493165 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7445f6d84-j452c"] Jul 17 10:54:04.560956 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.560919 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-944d58c6d-nrcts"] Jul 17 10:54:04.564155 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.564138 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.634087 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.634053 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-944d58c6d-nrcts"] Jul 17 10:54:04.643482 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.643453 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-service-ca\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.643482 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.643492 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfrst\" (UniqueName: \"kubernetes.io/projected/c44c5a37-b8b6-4e53-8410-7c674ad084a8-kube-api-access-vfrst\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.643734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.643567 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-serving-cert\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.643734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.643604 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-oauth-config\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.643734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.643650 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-oauth-serving-cert\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.643734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.643683 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-trusted-ca-bundle\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.643734 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.643703 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-config\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.745162 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.745123 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-oauth-serving-cert\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.745440 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.745417 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-trusted-ca-bundle\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.745520 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.745463 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-config\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.745616 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.745534 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-service-ca\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.745616 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.745597 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfrst\" (UniqueName: \"kubernetes.io/projected/c44c5a37-b8b6-4e53-8410-7c674ad084a8-kube-api-access-vfrst\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.745718 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.745651 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-serving-cert\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.745718 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.745679 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-oauth-config\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.746052 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.746025 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-oauth-serving-cert\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.746319 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.746284 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-config\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.746432 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.746313 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-service-ca\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.746499 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.746458 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-trusted-ca-bundle\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.748017 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.747999 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-oauth-config\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.748625 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.748605 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-serving-cert\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.760480 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.760448 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfrst\" (UniqueName: \"kubernetes.io/projected/c44c5a37-b8b6-4e53-8410-7c674ad084a8-kube-api-access-vfrst\") pod \"console-944d58c6d-nrcts\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:04.874236 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:04.874132 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:05.031806 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:05.031782 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-944d58c6d-nrcts"] Jul 17 10:54:05.034304 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:54:05.034271 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc44c5a37_b8b6_4e53_8410_7c674ad084a8.slice/crio-b697e2e72a7b41c039b8515d5e514ed5a6fd32391e2384f2dd8a7d79c6a83f6d WatchSource:0}: Error finding container b697e2e72a7b41c039b8515d5e514ed5a6fd32391e2384f2dd8a7d79c6a83f6d: Status 404 returned error can't find the container with id b697e2e72a7b41c039b8515d5e514ed5a6fd32391e2384f2dd8a7d79c6a83f6d Jul 17 10:54:05.317988 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:05.317953 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-944d58c6d-nrcts" event={"ID":"c44c5a37-b8b6-4e53-8410-7c674ad084a8","Type":"ContainerStarted","Data":"ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1"} Jul 17 10:54:05.318145 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:05.318006 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-944d58c6d-nrcts" event={"ID":"c44c5a37-b8b6-4e53-8410-7c674ad084a8","Type":"ContainerStarted","Data":"b697e2e72a7b41c039b8515d5e514ed5a6fd32391e2384f2dd8a7d79c6a83f6d"} Jul 17 10:54:05.353213 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:05.353152 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-944d58c6d-nrcts" podStartSLOduration=1.353135486 podStartE2EDuration="1.353135486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:54:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:54:05.352708697 +0000 UTC m=+109.026224360" watchObservedRunningTime="2026-07-17 10:54:05.353135486 +0000 UTC m=+109.026651138" Jul 17 10:54:07.129877 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.129839 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" podUID="4b8b6e43-da60-423a-9108-fda424fade1a" containerName="registry" containerID="cri-o://e7690649eb57462a830932381041ce2375dbeb30ca51941dbd3eafdda799bedb" gracePeriod=30 Jul 17 10:54:07.327064 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.327036 2551 generic.go:356] "Generic (PLEG): container finished" podID="4b8b6e43-da60-423a-9108-fda424fade1a" containerID="e7690649eb57462a830932381041ce2375dbeb30ca51941dbd3eafdda799bedb" exitCode=0 Jul 17 10:54:07.327206 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.327109 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" event={"ID":"4b8b6e43-da60-423a-9108-fda424fade1a","Type":"ContainerDied","Data":"e7690649eb57462a830932381041ce2375dbeb30ca51941dbd3eafdda799bedb"} Jul 17 10:54:07.334283 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.334256 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:54:07.376152 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.376129 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:54:07.472825 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.472729 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-installation-pull-secrets\") pod \"4b8b6e43-da60-423a-9108-fda424fade1a\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " Jul 17 10:54:07.472825 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.472772 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-image-registry-private-configuration\") pod \"4b8b6e43-da60-423a-9108-fda424fade1a\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " Jul 17 10:54:07.472825 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.472812 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-trusted-ca\") pod \"4b8b6e43-da60-423a-9108-fda424fade1a\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " Jul 17 10:54:07.473094 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.472842 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzfwc\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-kube-api-access-hzfwc\") pod \"4b8b6e43-da60-423a-9108-fda424fade1a\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " Jul 17 10:54:07.473094 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.472863 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b8b6e43-da60-423a-9108-fda424fade1a-ca-trust-extracted\") pod \"4b8b6e43-da60-423a-9108-fda424fade1a\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " Jul 17 10:54:07.473094 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.472904 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls\") pod \"4b8b6e43-da60-423a-9108-fda424fade1a\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " Jul 17 10:54:07.473094 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.472957 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-bound-sa-token\") pod \"4b8b6e43-da60-423a-9108-fda424fade1a\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " Jul 17 10:54:07.473094 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.472978 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-registry-certificates\") pod \"4b8b6e43-da60-423a-9108-fda424fade1a\" (UID: \"4b8b6e43-da60-423a-9108-fda424fade1a\") " Jul 17 10:54:07.473370 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.473345 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4b8b6e43-da60-423a-9108-fda424fade1a" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:54:07.473433 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.473369 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4b8b6e43-da60-423a-9108-fda424fade1a" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:54:07.473691 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.473668 2551 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-registry-certificates\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:07.473789 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.473696 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b8b6e43-da60-423a-9108-fda424fade1a-trusted-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:07.475503 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.475457 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4b8b6e43-da60-423a-9108-fda424fade1a" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:54:07.475686 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.475659 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4b8b6e43-da60-423a-9108-fda424fade1a" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:54:07.475686 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.475675 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4b8b6e43-da60-423a-9108-fda424fade1a" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 10:54:07.475797 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.475662 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-kube-api-access-hzfwc" (OuterVolumeSpecName: "kube-api-access-hzfwc") pod "4b8b6e43-da60-423a-9108-fda424fade1a" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a"). InnerVolumeSpecName "kube-api-access-hzfwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:54:07.475797 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.475773 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4b8b6e43-da60-423a-9108-fda424fade1a" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 10:54:07.483981 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.483952 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b8b6e43-da60-423a-9108-fda424fade1a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4b8b6e43-da60-423a-9108-fda424fade1a" (UID: "4b8b6e43-da60-423a-9108-fda424fade1a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:54:07.574420 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.574383 2551 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-bound-sa-token\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:07.574420 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.574413 2551 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-installation-pull-secrets\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:07.574420 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.574424 2551 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b8b6e43-da60-423a-9108-fda424fade1a-image-registry-private-configuration\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:07.574667 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.574435 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzfwc\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-kube-api-access-hzfwc\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:07.574667 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.574444 2551 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b8b6e43-da60-423a-9108-fda424fade1a-ca-trust-extracted\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:07.574667 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:07.574452 2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b8b6e43-da60-423a-9108-fda424fade1a-registry-tls\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:08.332385 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:08.332350 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" Jul 17 10:54:08.332385 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:08.332356 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b5c697c4d-rz846" event={"ID":"4b8b6e43-da60-423a-9108-fda424fade1a","Type":"ContainerDied","Data":"b16c9fa4f88fd29ba7e813eec9acb212bb92c1307b94537f3210f9ad2e105a1e"} Jul 17 10:54:08.332866 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:08.332417 2551 scope.go:117] "RemoveContainer" containerID="e7690649eb57462a830932381041ce2375dbeb30ca51941dbd3eafdda799bedb" Jul 17 10:54:08.355124 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:08.355095 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b5c697c4d-rz846"] Jul 17 10:54:08.359680 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:08.359657 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b5c697c4d-rz846"] Jul 17 10:54:08.821466 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:08.821436 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b8b6e43-da60-423a-9108-fda424fade1a" path="/var/lib/kubelet/pods/4b8b6e43-da60-423a-9108-fda424fade1a/volumes" Jul 17 10:54:10.301367 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:10.301340 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-66wkj" Jul 17 10:54:14.874822 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:14.874782 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:14.875324 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:14.874851 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:14.881069 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:14.881046 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:15.361209 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:15.361183 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:54:15.408954 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:15.408914 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b4b7f87d6-p5k48"] Jul 17 10:54:26.739077 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:26.739040 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:54:26.741258 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:26.741239 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9bb4909-4714-4484-96fb-e258962dfd4b-metrics-certs\") pod \"network-metrics-daemon-sqv6x\" (UID: \"c9bb4909-4714-4484-96fb-e258962dfd4b\") " pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:54:26.745819 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:26.745801 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rk769\"" Jul 17 10:54:26.753649 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:26.753625 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sqv6x" Jul 17 10:54:26.937800 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:26.937764 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sqv6x"] Jul 17 10:54:26.940518 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:54:26.940476 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9bb4909_4714_4484_96fb_e258962dfd4b.slice/crio-fcb00bd7e98ece99636fcf93e3e2b432842b89ed0856545794c387f0c6b547c2 WatchSource:0}: Error finding container fcb00bd7e98ece99636fcf93e3e2b432842b89ed0856545794c387f0c6b547c2: Status 404 returned error can't find the container with id fcb00bd7e98ece99636fcf93e3e2b432842b89ed0856545794c387f0c6b547c2 Jul 17 10:54:27.401683 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:27.401643 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sqv6x" event={"ID":"c9bb4909-4714-4484-96fb-e258962dfd4b","Type":"ContainerStarted","Data":"fcb00bd7e98ece99636fcf93e3e2b432842b89ed0856545794c387f0c6b547c2"} Jul 17 10:54:29.411866 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.411778 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sqv6x" event={"ID":"c9bb4909-4714-4484-96fb-e258962dfd4b","Type":"ContainerStarted","Data":"dbf2b322ae94cd23a32882aa9be0db2a1ebc7313b8e6a603ac03b2c34d1fdc72"} Jul 17 10:54:29.411866 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.411838 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sqv6x" event={"ID":"c9bb4909-4714-4484-96fb-e258962dfd4b","Type":"ContainerStarted","Data":"f5e3c9407e3ae4b2e73aa2c3cd41b82119564aafe1e9a73afae371770b22f267"} Jul 17 10:54:29.413270 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.413241 2551 generic.go:356] "Generic (PLEG): container finished" podID="6c351f21-c45a-4d18-939d-5f4159aad6fb" containerID="2762df7347b269f19a45b84a992544080e4692a3879529467905f347cb539827" exitCode=0 Jul 17 10:54:29.413386 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.413305 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lxp5b" event={"ID":"6c351f21-c45a-4d18-939d-5f4159aad6fb","Type":"ContainerDied","Data":"2762df7347b269f19a45b84a992544080e4692a3879529467905f347cb539827"} Jul 17 10:54:29.413601 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.413588 2551 scope.go:117] "RemoveContainer" containerID="2762df7347b269f19a45b84a992544080e4692a3879529467905f347cb539827" Jul 17 10:54:29.430584 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.430513 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sqv6x" podStartSLOduration=130.833897117 podStartE2EDuration="2m12.430498422s" totalImagesPullingTime="1.596601305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:54:29.430040476 +0000 UTC m=+133.103556130" watchObservedRunningTime="2026-07-17 10:54:29.430498422 +0000 UTC m=+133.104014073" Jul 17 10:54:29.518617 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.518574 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7445f6d84-j452c" podUID="04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" containerName="console" containerID="cri-o://ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985" gracePeriod=15 Jul 17 10:54:29.767140 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.767115 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:54:29.867478 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.867433 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64f58\" (UniqueName: \"kubernetes.io/projected/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-kube-api-access-64f58\") pod \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " Jul 17 10:54:29.867697 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.867511 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-config\") pod \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " Jul 17 10:54:29.867697 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.867537 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-oauth-config\") pod \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " Jul 17 10:54:29.867697 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.867627 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-oauth-serving-cert\") pod \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " Jul 17 10:54:29.867697 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.867655 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-trusted-ca-bundle\") pod \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " Jul 17 10:54:29.867915 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.867701 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-service-ca\") pod \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " Jul 17 10:54:29.867915 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.867724 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-serving-cert\") pod \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\" (UID: \"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa\") " Jul 17 10:54:29.868195 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.868165 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" (UID: "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:54:29.868281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.868185 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" (UID: "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:54:29.868281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.868200 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-service-ca" (OuterVolumeSpecName: "service-ca") pod "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" (UID: "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:54:29.868281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.868176 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-config" (OuterVolumeSpecName: "console-config") pod "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" (UID: "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:54:29.870034 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.870011 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" (UID: "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 10:54:29.870170 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.870148 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" (UID: "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 10:54:29.870233 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.870154 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-kube-api-access-64f58" (OuterVolumeSpecName: "kube-api-access-64f58") pod "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" (UID: "04a71433-3be4-4dfe-b5c4-0bf2ce4896aa"). InnerVolumeSpecName "kube-api-access-64f58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:54:29.968351 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.968257 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-oauth-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:29.968351 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.968285 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-trusted-ca-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:29.968351 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.968296 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:29.968351 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.968307 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:29.968351 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.968316 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64f58\" (UniqueName: \"kubernetes.io/projected/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-kube-api-access-64f58\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:29.968351 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.968326 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:29.968351 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:29.968334 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa-console-oauth-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:30.419528 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:30.419487 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lxp5b" event={"ID":"6c351f21-c45a-4d18-939d-5f4159aad6fb","Type":"ContainerStarted","Data":"1cc692039c810609f025697fcda13ea0913ea804eb2b5ccc84eb390756f25ffd"} Jul 17 10:54:30.420746 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:30.420720 2551 generic.go:356] "Generic (PLEG): container finished" podID="04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" containerID="ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985" exitCode=0 Jul 17 10:54:30.420846 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:30.420768 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7445f6d84-j452c" Jul 17 10:54:30.420846 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:30.420799 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7445f6d84-j452c" event={"ID":"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa","Type":"ContainerDied","Data":"ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985"} Jul 17 10:54:30.420934 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:30.420852 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7445f6d84-j452c" event={"ID":"04a71433-3be4-4dfe-b5c4-0bf2ce4896aa","Type":"ContainerDied","Data":"c8202482f405d1f6a9f76143c933001a7ff706330876b805bd402f038f58373b"} Jul 17 10:54:30.420934 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:30.420880 2551 scope.go:117] "RemoveContainer" containerID="ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985" Jul 17 10:54:30.430289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:30.430272 2551 scope.go:117] "RemoveContainer" containerID="ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985" Jul 17 10:54:30.430574 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:54:30.430525 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985\": container with ID starting with ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985 not found: ID does not exist" containerID="ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985" Jul 17 10:54:30.430666 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:30.430578 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985"} err="failed to get container status \"ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985\": rpc error: code = NotFound desc = could not find container \"ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985\": container with ID starting with ab067fb8d09ad42ccbe69c662084ed9bfbb986ccc4a3814251549c0b1ca05985 not found: ID does not exist" Jul 17 10:54:30.449968 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:30.449941 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7445f6d84-j452c"] Jul 17 10:54:30.458084 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:30.458059 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7445f6d84-j452c"] Jul 17 10:54:30.824693 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:30.824660 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" path="/var/lib/kubelet/pods/04a71433-3be4-4dfe-b5c4-0bf2ce4896aa/volumes" Jul 17 10:54:40.429232 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.429170 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b4b7f87d6-p5k48" podUID="962b5236-4918-487b-a657-fd99d9224722" containerName="console" containerID="cri-o://73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad" gracePeriod=15 Jul 17 10:54:40.667900 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.667874 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:54:40.746448 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.746356 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-console-config\") pod \"962b5236-4918-487b-a657-fd99d9224722\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " Jul 17 10:54:40.746448 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.746406 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-oauth-serving-cert\") pod \"962b5236-4918-487b-a657-fd99d9224722\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " Jul 17 10:54:40.746448 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.746442 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx2t5\" (UniqueName: \"kubernetes.io/projected/962b5236-4918-487b-a657-fd99d9224722-kube-api-access-tx2t5\") pod \"962b5236-4918-487b-a657-fd99d9224722\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " Jul 17 10:54:40.746744 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.746471 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-oauth-config\") pod \"962b5236-4918-487b-a657-fd99d9224722\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " Jul 17 10:54:40.746744 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.746501 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-service-ca\") pod \"962b5236-4918-487b-a657-fd99d9224722\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " Jul 17 10:54:40.746744 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.746537 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-serving-cert\") pod \"962b5236-4918-487b-a657-fd99d9224722\" (UID: \"962b5236-4918-487b-a657-fd99d9224722\") " Jul 17 10:54:40.746955 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.746828 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-console-config" (OuterVolumeSpecName: "console-config") pod "962b5236-4918-487b-a657-fd99d9224722" (UID: "962b5236-4918-487b-a657-fd99d9224722"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:54:40.747094 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.746992 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "962b5236-4918-487b-a657-fd99d9224722" (UID: "962b5236-4918-487b-a657-fd99d9224722"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:54:40.747094 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.747010 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-service-ca" (OuterVolumeSpecName: "service-ca") pod "962b5236-4918-487b-a657-fd99d9224722" (UID: "962b5236-4918-487b-a657-fd99d9224722"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:54:40.748790 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.748761 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/962b5236-4918-487b-a657-fd99d9224722-kube-api-access-tx2t5" (OuterVolumeSpecName: "kube-api-access-tx2t5") pod "962b5236-4918-487b-a657-fd99d9224722" (UID: "962b5236-4918-487b-a657-fd99d9224722"). InnerVolumeSpecName "kube-api-access-tx2t5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:54:40.748790 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.748760 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "962b5236-4918-487b-a657-fd99d9224722" (UID: "962b5236-4918-487b-a657-fd99d9224722"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 10:54:40.748922 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.748833 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "962b5236-4918-487b-a657-fd99d9224722" (UID: "962b5236-4918-487b-a657-fd99d9224722"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 10:54:40.847711 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.847682 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-console-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:40.847711 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.847707 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-oauth-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:40.847711 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.847718 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx2t5\" (UniqueName: \"kubernetes.io/projected/962b5236-4918-487b-a657-fd99d9224722-kube-api-access-tx2t5\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:40.847909 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.847728 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-oauth-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:40.847909 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.847736 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/962b5236-4918-487b-a657-fd99d9224722-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:40.847909 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:40.847744 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/962b5236-4918-487b-a657-fd99d9224722-console-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:54:41.464495 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:41.464457 2551 generic.go:356] "Generic (PLEG): container finished" podID="962b5236-4918-487b-a657-fd99d9224722" containerID="73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad" exitCode=0 Jul 17 10:54:41.464942 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:41.464501 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4b7f87d6-p5k48" Jul 17 10:54:41.464942 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:41.464558 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4b7f87d6-p5k48" event={"ID":"962b5236-4918-487b-a657-fd99d9224722","Type":"ContainerDied","Data":"73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad"} Jul 17 10:54:41.464942 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:41.464612 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4b7f87d6-p5k48" event={"ID":"962b5236-4918-487b-a657-fd99d9224722","Type":"ContainerDied","Data":"b49bc36ccb45691844e517b1c66b69d7063f33bfe55a67928c2dc318d953a289"} Jul 17 10:54:41.464942 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:41.464632 2551 scope.go:117] "RemoveContainer" containerID="73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad" Jul 17 10:54:41.472448 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:41.472425 2551 scope.go:117] "RemoveContainer" containerID="73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad" Jul 17 10:54:41.472826 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:54:41.472807 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad\": container with ID starting with 73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad not found: ID does not exist" containerID="73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad" Jul 17 10:54:41.472895 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:41.472835 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad"} err="failed to get container status \"73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad\": rpc error: code = NotFound desc = could not find container \"73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad\": container with ID starting with 73199c95191e13e0b2328999008eac21de976b71a0fab6532eeaa566c6da71ad not found: ID does not exist" Jul 17 10:54:41.482223 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:41.482198 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b4b7f87d6-p5k48"] Jul 17 10:54:41.486526 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:41.486506 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b4b7f87d6-p5k48"] Jul 17 10:54:42.824999 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:54:42.824971 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="962b5236-4918-487b-a657-fd99d9224722" path="/var/lib/kubelet/pods/962b5236-4918-487b-a657-fd99d9224722/volumes" Jul 17 10:55:21.376605 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.376565 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f7dcf7857-bdqb4"] Jul 17 10:55:21.377105 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.376975 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="962b5236-4918-487b-a657-fd99d9224722" containerName="console" Jul 17 10:55:21.377105 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.376996 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="962b5236-4918-487b-a657-fd99d9224722" containerName="console" Jul 17 10:55:21.377105 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.377010 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" containerName="console" Jul 17 10:55:21.377105 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.377018 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" containerName="console" Jul 17 10:55:21.377105 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.377035 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b8b6e43-da60-423a-9108-fda424fade1a" containerName="registry" Jul 17 10:55:21.377105 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.377044 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b8b6e43-da60-423a-9108-fda424fade1a" containerName="registry" Jul 17 10:55:21.377390 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.377123 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="962b5236-4918-487b-a657-fd99d9224722" containerName="console" Jul 17 10:55:21.377390 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.377136 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="04a71433-3be4-4dfe-b5c4-0bf2ce4896aa" containerName="console" Jul 17 10:55:21.377390 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.377146 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b8b6e43-da60-423a-9108-fda424fade1a" containerName="registry" Jul 17 10:55:21.381438 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.381417 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.391246 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.391224 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7dcf7857-bdqb4"] Jul 17 10:55:21.471668 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.471625 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8sm9\" (UniqueName: \"kubernetes.io/projected/381dd4dc-a5b4-41e0-9240-bc82b1594498-kube-api-access-w8sm9\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.471668 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.471665 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-config\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.471890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.471688 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-service-ca\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.471890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.471748 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-oauth-config\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.471890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.471789 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-oauth-serving-cert\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.471890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.471807 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-serving-cert\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.471890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.471834 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-trusted-ca-bundle\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.572679 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.572640 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8sm9\" (UniqueName: \"kubernetes.io/projected/381dd4dc-a5b4-41e0-9240-bc82b1594498-kube-api-access-w8sm9\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.572679 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.572681 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-config\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.572931 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.572702 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-service-ca\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.572931 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.572828 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-oauth-config\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.572931 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.572872 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-oauth-serving-cert\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.572931 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.572898 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-serving-cert\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.573125 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.572938 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-trusted-ca-bundle\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.573504 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.573478 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-service-ca\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.573690 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.573495 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-config\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.573690 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.573526 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-oauth-serving-cert\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.573824 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.573747 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-trusted-ca-bundle\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.575328 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.575306 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-serving-cert\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.575424 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.575354 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-oauth-config\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.581766 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.581744 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8sm9\" (UniqueName: \"kubernetes.io/projected/381dd4dc-a5b4-41e0-9240-bc82b1594498-kube-api-access-w8sm9\") pod \"console-5f7dcf7857-bdqb4\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.692063 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.691973 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:21.824481 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:21.824320 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7dcf7857-bdqb4"] Jul 17 10:55:21.827017 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:55:21.826984 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod381dd4dc_a5b4_41e0_9240_bc82b1594498.slice/crio-676393e725e06d92ef9d41b5d4efe017c7216cfac731633a1c45f5cc192bf1a6 WatchSource:0}: Error finding container 676393e725e06d92ef9d41b5d4efe017c7216cfac731633a1c45f5cc192bf1a6: Status 404 returned error can't find the container with id 676393e725e06d92ef9d41b5d4efe017c7216cfac731633a1c45f5cc192bf1a6 Jul 17 10:55:22.619945 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:22.619912 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcf7857-bdqb4" event={"ID":"381dd4dc-a5b4-41e0-9240-bc82b1594498","Type":"ContainerStarted","Data":"cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884"} Jul 17 10:55:22.620392 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:22.619961 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcf7857-bdqb4" event={"ID":"381dd4dc-a5b4-41e0-9240-bc82b1594498","Type":"ContainerStarted","Data":"676393e725e06d92ef9d41b5d4efe017c7216cfac731633a1c45f5cc192bf1a6"} Jul 17 10:55:22.638911 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:22.638866 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f7dcf7857-bdqb4" podStartSLOduration=1.6388518840000001 podStartE2EDuration="1.638851884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:55:22.637835467 +0000 UTC m=+186.311351122" watchObservedRunningTime="2026-07-17 10:55:22.638851884 +0000 UTC m=+186.312367535" Jul 17 10:55:31.692481 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:31.692441 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:31.693001 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:31.692598 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:31.697590 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:31.697568 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:32.662337 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:32.662313 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:55:32.709983 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:32.709950 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-944d58c6d-nrcts"] Jul 17 10:55:57.731033 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:57.730988 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-944d58c6d-nrcts" podUID="c44c5a37-b8b6-4e53-8410-7c674ad084a8" containerName="console" containerID="cri-o://ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1" gracePeriod=15 Jul 17 10:55:57.978469 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:57.978445 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:55:58.061181 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.061138 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-oauth-serving-cert\") pod \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " Jul 17 10:55:58.061365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.061203 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-serving-cert\") pod \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " Jul 17 10:55:58.061365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.061229 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-config\") pod \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " Jul 17 10:55:58.061365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.061258 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-trusted-ca-bundle\") pod \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " Jul 17 10:55:58.061365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.061301 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-oauth-config\") pod \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " Jul 17 10:55:58.061365 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.061349 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-service-ca\") pod \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " Jul 17 10:55:58.061646 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.061374 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfrst\" (UniqueName: \"kubernetes.io/projected/c44c5a37-b8b6-4e53-8410-7c674ad084a8-kube-api-access-vfrst\") pod \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\" (UID: \"c44c5a37-b8b6-4e53-8410-7c674ad084a8\") " Jul 17 10:55:58.061710 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.061621 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-config" (OuterVolumeSpecName: "console-config") pod "c44c5a37-b8b6-4e53-8410-7c674ad084a8" (UID: "c44c5a37-b8b6-4e53-8410-7c674ad084a8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:55:58.061856 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.061833 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c44c5a37-b8b6-4e53-8410-7c674ad084a8" (UID: "c44c5a37-b8b6-4e53-8410-7c674ad084a8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:55:58.061928 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.061847 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c44c5a37-b8b6-4e53-8410-7c674ad084a8" (UID: "c44c5a37-b8b6-4e53-8410-7c674ad084a8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:55:58.061928 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.061829 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-service-ca" (OuterVolumeSpecName: "service-ca") pod "c44c5a37-b8b6-4e53-8410-7c674ad084a8" (UID: "c44c5a37-b8b6-4e53-8410-7c674ad084a8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:55:58.063412 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.063379 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c44c5a37-b8b6-4e53-8410-7c674ad084a8" (UID: "c44c5a37-b8b6-4e53-8410-7c674ad084a8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 10:55:58.063521 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.063496 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c44c5a37-b8b6-4e53-8410-7c674ad084a8-kube-api-access-vfrst" (OuterVolumeSpecName: "kube-api-access-vfrst") pod "c44c5a37-b8b6-4e53-8410-7c674ad084a8" (UID: "c44c5a37-b8b6-4e53-8410-7c674ad084a8"). InnerVolumeSpecName "kube-api-access-vfrst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:55:58.063596 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.063525 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c44c5a37-b8b6-4e53-8410-7c674ad084a8" (UID: "c44c5a37-b8b6-4e53-8410-7c674ad084a8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 10:55:58.162111 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.162060 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-trusted-ca-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:55:58.162111 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.162106 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-oauth-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:55:58.162111 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.162117 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:55:58.162338 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.162136 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfrst\" (UniqueName: \"kubernetes.io/projected/c44c5a37-b8b6-4e53-8410-7c674ad084a8-kube-api-access-vfrst\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:55:58.162338 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.162146 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-oauth-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:55:58.162338 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.162154 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:55:58.162338 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.162164 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c44c5a37-b8b6-4e53-8410-7c674ad084a8-console-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:55:58.751425 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.751382 2551 generic.go:356] "Generic (PLEG): container finished" podID="c44c5a37-b8b6-4e53-8410-7c674ad084a8" containerID="ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1" exitCode=0 Jul 17 10:55:58.751425 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.751425 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-944d58c6d-nrcts" Jul 17 10:55:58.751935 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.751433 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-944d58c6d-nrcts" event={"ID":"c44c5a37-b8b6-4e53-8410-7c674ad084a8","Type":"ContainerDied","Data":"ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1"} Jul 17 10:55:58.751935 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.751483 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-944d58c6d-nrcts" event={"ID":"c44c5a37-b8b6-4e53-8410-7c674ad084a8","Type":"ContainerDied","Data":"b697e2e72a7b41c039b8515d5e514ed5a6fd32391e2384f2dd8a7d79c6a83f6d"} Jul 17 10:55:58.751935 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.751510 2551 scope.go:117] "RemoveContainer" containerID="ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1" Jul 17 10:55:58.760461 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.760442 2551 scope.go:117] "RemoveContainer" containerID="ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1" Jul 17 10:55:58.760801 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:55:58.760772 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1\": container with ID starting with ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1 not found: ID does not exist" containerID="ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1" Jul 17 10:55:58.760898 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.760811 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1"} err="failed to get container status \"ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1\": rpc error: code = NotFound desc = could not find container \"ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1\": container with ID starting with ca7222eebae4eab10c244ecc825b0c2cc5cb8028438db330d87e1d350957f5d1 not found: ID does not exist" Jul 17 10:55:58.774333 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.774306 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-944d58c6d-nrcts"] Jul 17 10:55:58.778781 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.778761 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-944d58c6d-nrcts"] Jul 17 10:55:58.822737 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:55:58.822709 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c44c5a37-b8b6-4e53-8410-7c674ad084a8" path="/var/lib/kubelet/pods/c44c5a37-b8b6-4e53-8410-7c674ad084a8/volumes" Jul 17 10:56:35.583055 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.583010 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-798599678b-sjn6n"] Jul 17 10:56:35.583639 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.583398 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c44c5a37-b8b6-4e53-8410-7c674ad084a8" containerName="console" Jul 17 10:56:35.583639 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.583416 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c44c5a37-b8b6-4e53-8410-7c674ad084a8" containerName="console" Jul 17 10:56:35.583639 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.583509 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c44c5a37-b8b6-4e53-8410-7c674ad084a8" containerName="console" Jul 17 10:56:35.587666 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.587642 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.599496 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.599473 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-798599678b-sjn6n"] Jul 17 10:56:35.758836 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.758795 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-oauth-config\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.758836 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.758832 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-config\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.759055 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.758857 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-serving-cert\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.759055 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.758934 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-trusted-ca-bundle\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.759055 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.758969 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbn2n\" (UniqueName: \"kubernetes.io/projected/68a2a055-8a30-41d9-92bd-1577defc3fe7-kube-api-access-kbn2n\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.759055 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.759003 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-service-ca\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.759055 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.759024 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-oauth-serving-cert\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.860001 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.859904 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-service-ca\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.860001 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.859949 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-oauth-serving-cert\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.860001 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.859969 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-oauth-config\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.860001 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.859990 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-config\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.860322 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.860024 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-serving-cert\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.860322 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.860245 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-trusted-ca-bundle\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.860322 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.860319 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbn2n\" (UniqueName: \"kubernetes.io/projected/68a2a055-8a30-41d9-92bd-1577defc3fe7-kube-api-access-kbn2n\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.860810 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.860785 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-service-ca\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.860970 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.860789 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-config\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.860970 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.860849 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-oauth-serving-cert\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.861089 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.861058 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-trusted-ca-bundle\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.862435 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.862417 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-oauth-config\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.862662 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.862642 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-serving-cert\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.868923 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.868904 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbn2n\" (UniqueName: \"kubernetes.io/projected/68a2a055-8a30-41d9-92bd-1577defc3fe7-kube-api-access-kbn2n\") pod \"console-798599678b-sjn6n\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:35.898053 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:35.898028 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:36.030105 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:36.030045 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-798599678b-sjn6n"] Jul 17 10:56:36.032245 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:56:36.032218 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a2a055_8a30_41d9_92bd_1577defc3fe7.slice/crio-c7fd9ed6ccc5deb5fd3dbb9a68d267d9b2a3ae315cb0ccc88535a9be392b8efe WatchSource:0}: Error finding container c7fd9ed6ccc5deb5fd3dbb9a68d267d9b2a3ae315cb0ccc88535a9be392b8efe: Status 404 returned error can't find the container with id c7fd9ed6ccc5deb5fd3dbb9a68d267d9b2a3ae315cb0ccc88535a9be392b8efe Jul 17 10:56:36.889330 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:36.889291 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798599678b-sjn6n" event={"ID":"68a2a055-8a30-41d9-92bd-1577defc3fe7","Type":"ContainerStarted","Data":"0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4"} Jul 17 10:56:36.889781 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:36.889348 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798599678b-sjn6n" event={"ID":"68a2a055-8a30-41d9-92bd-1577defc3fe7","Type":"ContainerStarted","Data":"c7fd9ed6ccc5deb5fd3dbb9a68d267d9b2a3ae315cb0ccc88535a9be392b8efe"} Jul 17 10:56:36.909243 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:36.909199 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-798599678b-sjn6n" podStartSLOduration=1.9091816069999998 podStartE2EDuration="1.909181607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:56:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:56:36.908071722 +0000 UTC m=+260.581587374" watchObservedRunningTime="2026-07-17 10:56:36.909181607 +0000 UTC m=+260.582697259" Jul 17 10:56:45.898538 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:45.898445 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:45.898538 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:45.898522 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:45.903528 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:45.903506 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:45.924158 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:45.924135 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-798599678b-sjn6n" Jul 17 10:56:45.969536 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:56:45.969500 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7dcf7857-bdqb4"] Jul 17 10:57:02.555957 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.555907 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x"] Jul 17 10:57:02.559402 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.559384 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:02.562210 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.562186 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 10:57:02.563241 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.563222 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zqgnn\"" Jul 17 10:57:02.563352 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.563260 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 10:57:02.568050 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.568026 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x"] Jul 17 10:57:02.659637 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.659597 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:02.659815 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.659680 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:02.659815 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.659745 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frmnt\" (UniqueName: \"kubernetes.io/projected/fdd00777-c822-4d60-91a1-357caa753921-kube-api-access-frmnt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:02.760568 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.760507 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:02.760568 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.760578 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:02.760781 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.760618 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frmnt\" (UniqueName: \"kubernetes.io/projected/fdd00777-c822-4d60-91a1-357caa753921-kube-api-access-frmnt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:02.761038 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.761015 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:02.761078 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.761025 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:02.769731 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.769701 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frmnt\" (UniqueName: \"kubernetes.io/projected/fdd00777-c822-4d60-91a1-357caa753921-kube-api-access-frmnt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:02.869801 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.869712 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:02.995860 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:02.995832 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x"] Jul 17 10:57:02.998369 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:57:02.998340 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd00777_c822_4d60_91a1_357caa753921.slice/crio-cc40fcde72d472867f0441c65a9ec9df3f43ecb876b04c00fa85729b1aaf4d5e WatchSource:0}: Error finding container cc40fcde72d472867f0441c65a9ec9df3f43ecb876b04c00fa85729b1aaf4d5e: Status 404 returned error can't find the container with id cc40fcde72d472867f0441c65a9ec9df3f43ecb876b04c00fa85729b1aaf4d5e Jul 17 10:57:03.986627 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:03.986588 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" event={"ID":"fdd00777-c822-4d60-91a1-357caa753921","Type":"ContainerStarted","Data":"cc40fcde72d472867f0441c65a9ec9df3f43ecb876b04c00fa85729b1aaf4d5e"} Jul 17 10:57:10.011464 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:10.011426 2551 generic.go:356] "Generic (PLEG): container finished" podID="fdd00777-c822-4d60-91a1-357caa753921" containerID="cae85beb154a1718048175e322f2f6404d6ddcb391c64ff9c3afec92d4f32c13" exitCode=0 Jul 17 10:57:10.011883 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:10.011498 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" event={"ID":"fdd00777-c822-4d60-91a1-357caa753921","Type":"ContainerDied","Data":"cae85beb154a1718048175e322f2f6404d6ddcb391c64ff9c3afec92d4f32c13"} Jul 17 10:57:10.996562 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:10.996513 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f7dcf7857-bdqb4" podUID="381dd4dc-a5b4-41e0-9240-bc82b1594498" containerName="console" containerID="cri-o://cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884" gracePeriod=15 Jul 17 10:57:11.260851 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.260828 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:57:11.331821 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.331785 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-config\") pod \"381dd4dc-a5b4-41e0-9240-bc82b1594498\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " Jul 17 10:57:11.332005 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.331845 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-serving-cert\") pod \"381dd4dc-a5b4-41e0-9240-bc82b1594498\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " Jul 17 10:57:11.332005 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.331875 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-trusted-ca-bundle\") pod \"381dd4dc-a5b4-41e0-9240-bc82b1594498\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " Jul 17 10:57:11.332005 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.331927 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-oauth-config\") pod \"381dd4dc-a5b4-41e0-9240-bc82b1594498\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " Jul 17 10:57:11.332005 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.331955 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-service-ca\") pod \"381dd4dc-a5b4-41e0-9240-bc82b1594498\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " Jul 17 10:57:11.332005 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.331975 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-oauth-serving-cert\") pod \"381dd4dc-a5b4-41e0-9240-bc82b1594498\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " Jul 17 10:57:11.332264 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.332033 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8sm9\" (UniqueName: \"kubernetes.io/projected/381dd4dc-a5b4-41e0-9240-bc82b1594498-kube-api-access-w8sm9\") pod \"381dd4dc-a5b4-41e0-9240-bc82b1594498\" (UID: \"381dd4dc-a5b4-41e0-9240-bc82b1594498\") " Jul 17 10:57:11.332319 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.332252 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-config" (OuterVolumeSpecName: "console-config") pod "381dd4dc-a5b4-41e0-9240-bc82b1594498" (UID: "381dd4dc-a5b4-41e0-9240-bc82b1594498"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:57:11.332699 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.332514 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-service-ca" (OuterVolumeSpecName: "service-ca") pod "381dd4dc-a5b4-41e0-9240-bc82b1594498" (UID: "381dd4dc-a5b4-41e0-9240-bc82b1594498"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:57:11.332699 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.332653 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "381dd4dc-a5b4-41e0-9240-bc82b1594498" (UID: "381dd4dc-a5b4-41e0-9240-bc82b1594498"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:57:11.332865 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.332717 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "381dd4dc-a5b4-41e0-9240-bc82b1594498" (UID: "381dd4dc-a5b4-41e0-9240-bc82b1594498"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 10:57:11.334387 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.334351 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "381dd4dc-a5b4-41e0-9240-bc82b1594498" (UID: "381dd4dc-a5b4-41e0-9240-bc82b1594498"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 10:57:11.334485 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.334436 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "381dd4dc-a5b4-41e0-9240-bc82b1594498" (UID: "381dd4dc-a5b4-41e0-9240-bc82b1594498"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 10:57:11.334575 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.334503 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/381dd4dc-a5b4-41e0-9240-bc82b1594498-kube-api-access-w8sm9" (OuterVolumeSpecName: "kube-api-access-w8sm9") pod "381dd4dc-a5b4-41e0-9240-bc82b1594498" (UID: "381dd4dc-a5b4-41e0-9240-bc82b1594498"). InnerVolumeSpecName "kube-api-access-w8sm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:57:11.432879 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.432845 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8sm9\" (UniqueName: \"kubernetes.io/projected/381dd4dc-a5b4-41e0-9240-bc82b1594498-kube-api-access-w8sm9\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:11.432879 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.432877 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:11.432879 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.432886 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:11.433109 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.432894 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-trusted-ca-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:11.433109 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.432903 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/381dd4dc-a5b4-41e0-9240-bc82b1594498-console-oauth-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:11.433109 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.432911 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:11.433109 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:11.432919 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/381dd4dc-a5b4-41e0-9240-bc82b1594498-oauth-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:12.021147 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:12.021112 2551 generic.go:356] "Generic (PLEG): container finished" podID="381dd4dc-a5b4-41e0-9240-bc82b1594498" containerID="cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884" exitCode=0 Jul 17 10:57:12.021309 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:12.021156 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7dcf7857-bdqb4" Jul 17 10:57:12.021309 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:12.021183 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcf7857-bdqb4" event={"ID":"381dd4dc-a5b4-41e0-9240-bc82b1594498","Type":"ContainerDied","Data":"cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884"} Jul 17 10:57:12.021309 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:12.021222 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7dcf7857-bdqb4" event={"ID":"381dd4dc-a5b4-41e0-9240-bc82b1594498","Type":"ContainerDied","Data":"676393e725e06d92ef9d41b5d4efe017c7216cfac731633a1c45f5cc192bf1a6"} Jul 17 10:57:12.021309 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:12.021240 2551 scope.go:117] "RemoveContainer" containerID="cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884" Jul 17 10:57:12.041920 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:12.041895 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7dcf7857-bdqb4"] Jul 17 10:57:12.046214 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:12.046192 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f7dcf7857-bdqb4"] Jul 17 10:57:12.079815 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:12.079795 2551 scope.go:117] "RemoveContainer" containerID="cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884" Jul 17 10:57:12.080130 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:57:12.080109 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884\": container with ID starting with cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884 not found: ID does not exist" containerID="cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884" Jul 17 10:57:12.080199 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:12.080140 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884"} err="failed to get container status \"cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884\": rpc error: code = NotFound desc = could not find container \"cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884\": container with ID starting with cad8142c0ec17a719878ce24042fc53d4ba1ec201140042a0f0e76ab78e70884 not found: ID does not exist" Jul 17 10:57:12.821981 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:12.821949 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="381dd4dc-a5b4-41e0-9240-bc82b1594498" path="/var/lib/kubelet/pods/381dd4dc-a5b4-41e0-9240-bc82b1594498/volumes" Jul 17 10:57:13.026964 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:13.026934 2551 generic.go:356] "Generic (PLEG): container finished" podID="fdd00777-c822-4d60-91a1-357caa753921" containerID="18a2a4914d28dadea7f18d8c57116d0d173b9144ae46d63d7e4773321e935ece" exitCode=0 Jul 17 10:57:13.027128 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:13.026977 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" event={"ID":"fdd00777-c822-4d60-91a1-357caa753921","Type":"ContainerDied","Data":"18a2a4914d28dadea7f18d8c57116d0d173b9144ae46d63d7e4773321e935ece"} Jul 17 10:57:16.767067 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:16.767018 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 10:57:16.768755 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:16.768733 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 10:57:16.779092 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:16.779071 2551 kubelet.go:1648] "Image garbage collection succeeded" Jul 17 10:57:21.059825 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:21.059789 2551 generic.go:356] "Generic (PLEG): container finished" podID="fdd00777-c822-4d60-91a1-357caa753921" containerID="a7c8a9324e97be21679d61dd659d48704ffde4fa6b084af3ade61298bcc525fb" exitCode=0 Jul 17 10:57:21.060212 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:21.059874 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" event={"ID":"fdd00777-c822-4d60-91a1-357caa753921","Type":"ContainerDied","Data":"a7c8a9324e97be21679d61dd659d48704ffde4fa6b084af3ade61298bcc525fb"} Jul 17 10:57:22.190150 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:22.190127 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:22.332187 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:22.332102 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-util\") pod \"fdd00777-c822-4d60-91a1-357caa753921\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " Jul 17 10:57:22.332187 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:22.332144 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frmnt\" (UniqueName: \"kubernetes.io/projected/fdd00777-c822-4d60-91a1-357caa753921-kube-api-access-frmnt\") pod \"fdd00777-c822-4d60-91a1-357caa753921\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " Jul 17 10:57:22.332414 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:22.332224 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-bundle\") pod \"fdd00777-c822-4d60-91a1-357caa753921\" (UID: \"fdd00777-c822-4d60-91a1-357caa753921\") " Jul 17 10:57:22.333080 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:22.333048 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-bundle" (OuterVolumeSpecName: "bundle") pod "fdd00777-c822-4d60-91a1-357caa753921" (UID: "fdd00777-c822-4d60-91a1-357caa753921"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:57:22.335024 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:22.334998 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdd00777-c822-4d60-91a1-357caa753921-kube-api-access-frmnt" (OuterVolumeSpecName: "kube-api-access-frmnt") pod "fdd00777-c822-4d60-91a1-357caa753921" (UID: "fdd00777-c822-4d60-91a1-357caa753921"). InnerVolumeSpecName "kube-api-access-frmnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:57:22.337249 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:22.337229 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-util" (OuterVolumeSpecName: "util") pod "fdd00777-c822-4d60-91a1-357caa753921" (UID: "fdd00777-c822-4d60-91a1-357caa753921"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:57:22.432918 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:22.432884 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:22.432918 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:22.432913 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frmnt\" (UniqueName: \"kubernetes.io/projected/fdd00777-c822-4d60-91a1-357caa753921-kube-api-access-frmnt\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:22.432918 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:22.432924 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdd00777-c822-4d60-91a1-357caa753921-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:23.068883 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:23.068843 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" event={"ID":"fdd00777-c822-4d60-91a1-357caa753921","Type":"ContainerDied","Data":"cc40fcde72d472867f0441c65a9ec9df3f43ecb876b04c00fa85729b1aaf4d5e"} Jul 17 10:57:23.068883 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:23.068872 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tff4x" Jul 17 10:57:23.069081 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:23.068899 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc40fcde72d472867f0441c65a9ec9df3f43ecb876b04c00fa85729b1aaf4d5e" Jul 17 10:57:30.556398 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.556363 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68"] Jul 17 10:57:30.556823 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.556646 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdd00777-c822-4d60-91a1-357caa753921" containerName="pull" Jul 17 10:57:30.556823 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.556657 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdd00777-c822-4d60-91a1-357caa753921" containerName="pull" Jul 17 10:57:30.556823 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.556672 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdd00777-c822-4d60-91a1-357caa753921" containerName="util" Jul 17 10:57:30.556823 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.556677 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdd00777-c822-4d60-91a1-357caa753921" containerName="util" Jul 17 10:57:30.556823 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.556693 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="381dd4dc-a5b4-41e0-9240-bc82b1594498" containerName="console" Jul 17 10:57:30.556823 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.556698 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="381dd4dc-a5b4-41e0-9240-bc82b1594498" containerName="console" Jul 17 10:57:30.556823 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.556706 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdd00777-c822-4d60-91a1-357caa753921" containerName="extract" Jul 17 10:57:30.556823 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.556711 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdd00777-c822-4d60-91a1-357caa753921" containerName="extract" Jul 17 10:57:30.556823 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.556765 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdd00777-c822-4d60-91a1-357caa753921" containerName="extract" Jul 17 10:57:30.556823 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.556780 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="381dd4dc-a5b4-41e0-9240-bc82b1594498" containerName="console" Jul 17 10:57:30.589659 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.589633 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68"] Jul 17 10:57:30.589817 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.589727 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68" Jul 17 10:57:30.592303 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.592282 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 17 10:57:30.592402 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.592345 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 17 10:57:30.592402 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.592383 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-dtld8\"" Jul 17 10:57:30.693935 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.693900 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8a140cd-059e-4e4e-b5f1-614e4434e770-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9ml68\" (UID: \"b8a140cd-059e-4e4e-b5f1-614e4434e770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68" Jul 17 10:57:30.694100 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.694022 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztbl2\" (UniqueName: \"kubernetes.io/projected/b8a140cd-059e-4e4e-b5f1-614e4434e770-kube-api-access-ztbl2\") pod \"cert-manager-operator-controller-manager-55c444bf86-9ml68\" (UID: \"b8a140cd-059e-4e4e-b5f1-614e4434e770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68" Jul 17 10:57:30.795360 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.795319 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztbl2\" (UniqueName: \"kubernetes.io/projected/b8a140cd-059e-4e4e-b5f1-614e4434e770-kube-api-access-ztbl2\") pod \"cert-manager-operator-controller-manager-55c444bf86-9ml68\" (UID: \"b8a140cd-059e-4e4e-b5f1-614e4434e770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68" Jul 17 10:57:30.795502 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.795370 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8a140cd-059e-4e4e-b5f1-614e4434e770-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9ml68\" (UID: \"b8a140cd-059e-4e4e-b5f1-614e4434e770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68" Jul 17 10:57:30.795806 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.795787 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8a140cd-059e-4e4e-b5f1-614e4434e770-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9ml68\" (UID: \"b8a140cd-059e-4e4e-b5f1-614e4434e770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68" Jul 17 10:57:30.804097 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.804074 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztbl2\" (UniqueName: \"kubernetes.io/projected/b8a140cd-059e-4e4e-b5f1-614e4434e770-kube-api-access-ztbl2\") pod \"cert-manager-operator-controller-manager-55c444bf86-9ml68\" (UID: \"b8a140cd-059e-4e4e-b5f1-614e4434e770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68" Jul 17 10:57:30.898825 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:30.898749 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68" Jul 17 10:57:31.050798 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:31.050771 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68"] Jul 17 10:57:31.053131 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:57:31.053105 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a140cd_059e_4e4e_b5f1_614e4434e770.slice/crio-97732f60e417478ca8dad1d27bf5a9daabb82362219a2052357609d2bc9966ce WatchSource:0}: Error finding container 97732f60e417478ca8dad1d27bf5a9daabb82362219a2052357609d2bc9966ce: Status 404 returned error can't find the container with id 97732f60e417478ca8dad1d27bf5a9daabb82362219a2052357609d2bc9966ce Jul 17 10:57:31.055753 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:31.055737 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 10:57:31.099544 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:31.099512 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68" event={"ID":"b8a140cd-059e-4e4e-b5f1-614e4434e770","Type":"ContainerStarted","Data":"97732f60e417478ca8dad1d27bf5a9daabb82362219a2052357609d2bc9966ce"} Jul 17 10:57:34.030095 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.030058 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-phxxq"] Jul 17 10:57:34.032285 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.032269 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" Jul 17 10:57:34.034958 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.034933 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 17 10:57:34.035069 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.034986 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 17 10:57:34.043803 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.043781 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-phxxq"] Jul 17 10:57:34.113693 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.113657 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68" event={"ID":"b8a140cd-059e-4e4e-b5f1-614e4434e770","Type":"ContainerStarted","Data":"644dd5abbf66dcbf5c1f8610eb6067c06d3d83fe85fba8ba83d9154ee139e1f8"} Jul 17 10:57:34.123477 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.123449 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd318855-2c53-41bd-b45d-05b4fd8f98b7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-phxxq\" (UID: \"dd318855-2c53-41bd-b45d-05b4fd8f98b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" Jul 17 10:57:34.123626 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.123492 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2vdv\" (UniqueName: \"kubernetes.io/projected/dd318855-2c53-41bd-b45d-05b4fd8f98b7-kube-api-access-m2vdv\") pod \"cert-manager-webhook-6486ffd997-phxxq\" (UID: \"dd318855-2c53-41bd-b45d-05b4fd8f98b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" Jul 17 10:57:34.132820 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.132768 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9ml68" podStartSLOduration=2.10694582 podStartE2EDuration="4.132753447s" totalImagesPullingTime="2.025807627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:34.131734156 +0000 UTC m=+317.805249809" watchObservedRunningTime="2026-07-17 10:57:34.132753447 +0000 UTC m=+317.806269098" Jul 17 10:57:34.224330 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.224291 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd318855-2c53-41bd-b45d-05b4fd8f98b7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-phxxq\" (UID: \"dd318855-2c53-41bd-b45d-05b4fd8f98b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" Jul 17 10:57:34.224513 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.224350 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2vdv\" (UniqueName: \"kubernetes.io/projected/dd318855-2c53-41bd-b45d-05b4fd8f98b7-kube-api-access-m2vdv\") pod \"cert-manager-webhook-6486ffd997-phxxq\" (UID: \"dd318855-2c53-41bd-b45d-05b4fd8f98b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" Jul 17 10:57:34.232412 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.232385 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd318855-2c53-41bd-b45d-05b4fd8f98b7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-phxxq\" (UID: \"dd318855-2c53-41bd-b45d-05b4fd8f98b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" Jul 17 10:57:34.232576 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.232510 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2vdv\" (UniqueName: \"kubernetes.io/projected/dd318855-2c53-41bd-b45d-05b4fd8f98b7-kube-api-access-m2vdv\") pod \"cert-manager-webhook-6486ffd997-phxxq\" (UID: \"dd318855-2c53-41bd-b45d-05b4fd8f98b7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" Jul 17 10:57:34.350345 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.350252 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" Jul 17 10:57:34.491507 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:34.491478 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-phxxq"] Jul 17 10:57:34.493881 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:57:34.493843 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd318855_2c53_41bd_b45d_05b4fd8f98b7.slice/crio-e7105ab2a4fb5ebf4edfb353fb247289e5f7fbb5988f5ba2161e95a060ace31a WatchSource:0}: Error finding container e7105ab2a4fb5ebf4edfb353fb247289e5f7fbb5988f5ba2161e95a060ace31a: Status 404 returned error can't find the container with id e7105ab2a4fb5ebf4edfb353fb247289e5f7fbb5988f5ba2161e95a060ace31a Jul 17 10:57:35.098341 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.098300 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6"] Jul 17 10:57:35.101417 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.101393 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:35.104147 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.104123 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 10:57:35.104250 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.104123 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 10:57:35.104250 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.104240 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zqgnn\"" Jul 17 10:57:35.110765 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.110741 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6"] Jul 17 10:57:35.119854 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.119818 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" event={"ID":"dd318855-2c53-41bd-b45d-05b4fd8f98b7","Type":"ContainerStarted","Data":"e7105ab2a4fb5ebf4edfb353fb247289e5f7fbb5988f5ba2161e95a060ace31a"} Jul 17 10:57:35.233333 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.233299 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:35.233516 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.233407 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k68s\" (UniqueName: \"kubernetes.io/projected/5717a0f1-7f42-4212-9e50-e140fd255ade-kube-api-access-5k68s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:35.233516 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.233456 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:35.333889 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.333845 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:35.334054 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.333947 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k68s\" (UniqueName: \"kubernetes.io/projected/5717a0f1-7f42-4212-9e50-e140fd255ade-kube-api-access-5k68s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:35.334054 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.333993 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:35.334346 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.334317 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:35.334467 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.334376 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:35.342466 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.342438 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k68s\" (UniqueName: \"kubernetes.io/projected/5717a0f1-7f42-4212-9e50-e140fd255ade-kube-api-access-5k68s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:35.411043 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.410964 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:35.558621 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:35.558432 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6"] Jul 17 10:57:35.561149 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:57:35.561112 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5717a0f1_7f42_4212_9e50_e140fd255ade.slice/crio-ebf297aa1b7c151c5747a0797c224d65c67ad84e00a3e3144599f620e32c3d1a WatchSource:0}: Error finding container ebf297aa1b7c151c5747a0797c224d65c67ad84e00a3e3144599f620e32c3d1a: Status 404 returned error can't find the container with id ebf297aa1b7c151c5747a0797c224d65c67ad84e00a3e3144599f620e32c3d1a Jul 17 10:57:36.126393 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:36.126349 2551 generic.go:356] "Generic (PLEG): container finished" podID="5717a0f1-7f42-4212-9e50-e140fd255ade" containerID="9dd3a8bd8c871388c90f1612ff327b0a6b1af237613bf72e458abf9a9e35e66f" exitCode=0 Jul 17 10:57:36.126859 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:36.126432 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" event={"ID":"5717a0f1-7f42-4212-9e50-e140fd255ade","Type":"ContainerDied","Data":"9dd3a8bd8c871388c90f1612ff327b0a6b1af237613bf72e458abf9a9e35e66f"} Jul 17 10:57:36.126859 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:36.126501 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" event={"ID":"5717a0f1-7f42-4212-9e50-e140fd255ade","Type":"ContainerStarted","Data":"ebf297aa1b7c151c5747a0797c224d65c67ad84e00a3e3144599f620e32c3d1a"} Jul 17 10:57:37.204275 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.204233 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p5cws"] Jul 17 10:57:37.207429 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.207349 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p5cws" Jul 17 10:57:37.210027 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.209991 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-kkk59\"" Jul 17 10:57:37.217314 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.217293 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p5cws"] Jul 17 10:57:37.352406 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.352363 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4qcv\" (UniqueName: \"kubernetes.io/projected/55bb644a-e7f3-4459-a934-c86677a71239-kube-api-access-x4qcv\") pod \"cert-manager-cainjector-559cdd8c87-p5cws\" (UID: \"55bb644a-e7f3-4459-a934-c86677a71239\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p5cws" Jul 17 10:57:37.352542 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.352442 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55bb644a-e7f3-4459-a934-c86677a71239-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p5cws\" (UID: \"55bb644a-e7f3-4459-a934-c86677a71239\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p5cws" Jul 17 10:57:37.453752 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.453716 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4qcv\" (UniqueName: \"kubernetes.io/projected/55bb644a-e7f3-4459-a934-c86677a71239-kube-api-access-x4qcv\") pod \"cert-manager-cainjector-559cdd8c87-p5cws\" (UID: \"55bb644a-e7f3-4459-a934-c86677a71239\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p5cws" Jul 17 10:57:37.454027 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.453790 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55bb644a-e7f3-4459-a934-c86677a71239-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p5cws\" (UID: \"55bb644a-e7f3-4459-a934-c86677a71239\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p5cws" Jul 17 10:57:37.462257 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.462181 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55bb644a-e7f3-4459-a934-c86677a71239-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p5cws\" (UID: \"55bb644a-e7f3-4459-a934-c86677a71239\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p5cws" Jul 17 10:57:37.462702 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.462677 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4qcv\" (UniqueName: \"kubernetes.io/projected/55bb644a-e7f3-4459-a934-c86677a71239-kube-api-access-x4qcv\") pod \"cert-manager-cainjector-559cdd8c87-p5cws\" (UID: \"55bb644a-e7f3-4459-a934-c86677a71239\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p5cws" Jul 17 10:57:37.516460 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.516424 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p5cws" Jul 17 10:57:37.666211 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:37.666182 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p5cws"] Jul 17 10:57:37.668657 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:57:37.668621 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55bb644a_e7f3_4459_a934_c86677a71239.slice/crio-9ec1d767b051b82b8c1b7f246dc817e09a9d583206845dfa80e15841fd1e30cd WatchSource:0}: Error finding container 9ec1d767b051b82b8c1b7f246dc817e09a9d583206845dfa80e15841fd1e30cd: Status 404 returned error can't find the container with id 9ec1d767b051b82b8c1b7f246dc817e09a9d583206845dfa80e15841fd1e30cd Jul 17 10:57:38.138614 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:38.138573 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p5cws" event={"ID":"55bb644a-e7f3-4459-a934-c86677a71239","Type":"ContainerStarted","Data":"156b63079cbba9ad67176abc718ae572941c975199e93734f3bb343a5bb25664"} Jul 17 10:57:38.138796 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:38.138629 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p5cws" event={"ID":"55bb644a-e7f3-4459-a934-c86677a71239","Type":"ContainerStarted","Data":"9ec1d767b051b82b8c1b7f246dc817e09a9d583206845dfa80e15841fd1e30cd"} Jul 17 10:57:38.140753 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:38.140714 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" event={"ID":"dd318855-2c53-41bd-b45d-05b4fd8f98b7","Type":"ContainerStarted","Data":"8ce12e9ea798c61316afd6208d93f401cf947cb5490202fdfba50cba6964f6ac"} Jul 17 10:57:38.140900 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:38.140855 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" Jul 17 10:57:38.157956 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:38.157905 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p5cws" podStartSLOduration=1.157882449 podStartE2EDuration="1.157882449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:38.157219565 +0000 UTC m=+321.830735218" watchObservedRunningTime="2026-07-17 10:57:38.157882449 +0000 UTC m=+321.831398106" Jul 17 10:57:38.178358 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:38.178306 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" podStartSLOduration=1.417160803 podStartE2EDuration="4.178285183s" totalImagesPullingTime="2.76112438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:57:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:38.176751958 +0000 UTC m=+321.850267613" watchObservedRunningTime="2026-07-17 10:57:38.178285183 +0000 UTC m=+321.851800836" Jul 17 10:57:39.146294 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:39.146262 2551 generic.go:356] "Generic (PLEG): container finished" podID="5717a0f1-7f42-4212-9e50-e140fd255ade" containerID="6e2a78f29d143adf7fd1416d1db070b24f14e2e299c3793d25ae5fb02cb4dcfc" exitCode=0 Jul 17 10:57:39.146783 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:39.146341 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" event={"ID":"5717a0f1-7f42-4212-9e50-e140fd255ade","Type":"ContainerDied","Data":"6e2a78f29d143adf7fd1416d1db070b24f14e2e299c3793d25ae5fb02cb4dcfc"} Jul 17 10:57:40.152887 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:40.152850 2551 generic.go:356] "Generic (PLEG): container finished" podID="5717a0f1-7f42-4212-9e50-e140fd255ade" containerID="eb6b0f991dad68d96dfb0b62218d8f8293aa3f2eb403f0c654ece165fec5331c" exitCode=0 Jul 17 10:57:40.153259 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:40.152914 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" event={"ID":"5717a0f1-7f42-4212-9e50-e140fd255ade","Type":"ContainerDied","Data":"eb6b0f991dad68d96dfb0b62218d8f8293aa3f2eb403f0c654ece165fec5331c"} Jul 17 10:57:41.283060 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:41.283033 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:41.390922 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:41.390889 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5k68s\" (UniqueName: \"kubernetes.io/projected/5717a0f1-7f42-4212-9e50-e140fd255ade-kube-api-access-5k68s\") pod \"5717a0f1-7f42-4212-9e50-e140fd255ade\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " Jul 17 10:57:41.391111 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:41.390982 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-bundle\") pod \"5717a0f1-7f42-4212-9e50-e140fd255ade\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " Jul 17 10:57:41.391111 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:41.391041 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-util\") pod \"5717a0f1-7f42-4212-9e50-e140fd255ade\" (UID: \"5717a0f1-7f42-4212-9e50-e140fd255ade\") " Jul 17 10:57:41.391394 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:41.391370 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-bundle" (OuterVolumeSpecName: "bundle") pod "5717a0f1-7f42-4212-9e50-e140fd255ade" (UID: "5717a0f1-7f42-4212-9e50-e140fd255ade"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:57:41.393043 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:41.393018 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5717a0f1-7f42-4212-9e50-e140fd255ade-kube-api-access-5k68s" (OuterVolumeSpecName: "kube-api-access-5k68s") pod "5717a0f1-7f42-4212-9e50-e140fd255ade" (UID: "5717a0f1-7f42-4212-9e50-e140fd255ade"). InnerVolumeSpecName "kube-api-access-5k68s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:57:41.395346 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:41.395321 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-util" (OuterVolumeSpecName: "util") pod "5717a0f1-7f42-4212-9e50-e140fd255ade" (UID: "5717a0f1-7f42-4212-9e50-e140fd255ade"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:57:41.492531 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:41.492449 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5k68s\" (UniqueName: \"kubernetes.io/projected/5717a0f1-7f42-4212-9e50-e140fd255ade-kube-api-access-5k68s\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:41.492531 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:41.492478 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:41.492531 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:41.492487 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5717a0f1-7f42-4212-9e50-e140fd255ade-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:42.162889 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:42.162856 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" Jul 17 10:57:42.163062 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:42.162864 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw5qj6" event={"ID":"5717a0f1-7f42-4212-9e50-e140fd255ade","Type":"ContainerDied","Data":"ebf297aa1b7c151c5747a0797c224d65c67ad84e00a3e3144599f620e32c3d1a"} Jul 17 10:57:42.163062 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:42.162986 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebf297aa1b7c151c5747a0797c224d65c67ad84e00a3e3144599f620e32c3d1a" Jul 17 10:57:44.149179 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:44.149149 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-phxxq" Jul 17 10:57:47.629373 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.629339 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2"] Jul 17 10:57:47.629778 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.629653 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5717a0f1-7f42-4212-9e50-e140fd255ade" containerName="extract" Jul 17 10:57:47.629778 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.629665 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5717a0f1-7f42-4212-9e50-e140fd255ade" containerName="extract" Jul 17 10:57:47.629778 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.629677 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5717a0f1-7f42-4212-9e50-e140fd255ade" containerName="util" Jul 17 10:57:47.629778 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.629683 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5717a0f1-7f42-4212-9e50-e140fd255ade" containerName="util" Jul 17 10:57:47.629778 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.629699 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5717a0f1-7f42-4212-9e50-e140fd255ade" containerName="pull" Jul 17 10:57:47.629778 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.629704 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5717a0f1-7f42-4212-9e50-e140fd255ade" containerName="pull" Jul 17 10:57:47.629778 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.629754 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5717a0f1-7f42-4212-9e50-e140fd255ade" containerName="extract" Jul 17 10:57:47.633287 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.633268 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2" Jul 17 10:57:47.635811 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.635791 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 17 10:57:47.636057 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.636040 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 17 10:57:47.636857 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.636837 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-qwbg7\"" Jul 17 10:57:47.644065 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.644042 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2"] Jul 17 10:57:47.645843 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.645818 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfsm6\" (UniqueName: \"kubernetes.io/projected/2b5900ac-609d-42dd-a55e-9deb2b3053e6-kube-api-access-xfsm6\") pod \"openshift-lws-operator-fd8ccff4c-wjtf2\" (UID: \"2b5900ac-609d-42dd-a55e-9deb2b3053e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2" Jul 17 10:57:47.645952 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.645919 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b5900ac-609d-42dd-a55e-9deb2b3053e6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wjtf2\" (UID: \"2b5900ac-609d-42dd-a55e-9deb2b3053e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2" Jul 17 10:57:47.746574 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.746514 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b5900ac-609d-42dd-a55e-9deb2b3053e6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wjtf2\" (UID: \"2b5900ac-609d-42dd-a55e-9deb2b3053e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2" Jul 17 10:57:47.746760 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.746595 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfsm6\" (UniqueName: \"kubernetes.io/projected/2b5900ac-609d-42dd-a55e-9deb2b3053e6-kube-api-access-xfsm6\") pod \"openshift-lws-operator-fd8ccff4c-wjtf2\" (UID: \"2b5900ac-609d-42dd-a55e-9deb2b3053e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2" Jul 17 10:57:47.746899 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.746879 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b5900ac-609d-42dd-a55e-9deb2b3053e6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wjtf2\" (UID: \"2b5900ac-609d-42dd-a55e-9deb2b3053e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2" Jul 17 10:57:47.754451 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.754428 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfsm6\" (UniqueName: \"kubernetes.io/projected/2b5900ac-609d-42dd-a55e-9deb2b3053e6-kube-api-access-xfsm6\") pod \"openshift-lws-operator-fd8ccff4c-wjtf2\" (UID: \"2b5900ac-609d-42dd-a55e-9deb2b3053e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2" Jul 17 10:57:47.943581 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:47.943477 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2" Jul 17 10:57:48.076289 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:48.076255 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2"] Jul 17 10:57:48.080059 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:57:48.080030 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b5900ac_609d_42dd_a55e_9deb2b3053e6.slice/crio-19dabbc683f8d274e04699cb807df92704aa5e08f8fa76608977309bc583b452 WatchSource:0}: Error finding container 19dabbc683f8d274e04699cb807df92704aa5e08f8fa76608977309bc583b452: Status 404 returned error can't find the container with id 19dabbc683f8d274e04699cb807df92704aa5e08f8fa76608977309bc583b452 Jul 17 10:57:48.188616 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:48.188583 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2" event={"ID":"2b5900ac-609d-42dd-a55e-9deb2b3053e6","Type":"ContainerStarted","Data":"19dabbc683f8d274e04699cb807df92704aa5e08f8fa76608977309bc583b452"} Jul 17 10:57:50.199026 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:50.198931 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2" event={"ID":"2b5900ac-609d-42dd-a55e-9deb2b3053e6","Type":"ContainerStarted","Data":"add9eedf50d46c713bd57f674887fb46c0d314a37324bf29b26dc03c87c8d4eb"} Jul 17 10:57:50.217163 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:50.217112 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjtf2" podStartSLOduration=1.490344903 podStartE2EDuration="3.217097731s" totalImagesPullingTime="1.726752828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:57:50.21584053 +0000 UTC m=+333.889356182" watchObservedRunningTime="2026-07-17 10:57:50.217097731 +0000 UTC m=+333.890613451" Jul 17 10:57:55.289757 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.289719 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5"] Jul 17 10:57:55.293192 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.293173 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:55.295824 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.295800 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 10:57:55.295824 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.295814 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 10:57:55.297356 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.297335 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zqgnn\"" Jul 17 10:57:55.300863 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.300833 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxsqn\" (UniqueName: \"kubernetes.io/projected/10453eb3-c597-4309-9b4d-2badea6493be-kube-api-access-dxsqn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:55.300973 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.300906 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:55.301032 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.300972 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:55.302460 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.302438 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5"] Jul 17 10:57:55.401322 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.401283 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:55.401513 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.401344 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxsqn\" (UniqueName: \"kubernetes.io/projected/10453eb3-c597-4309-9b4d-2badea6493be-kube-api-access-dxsqn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:55.401513 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.401381 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:55.401776 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.401759 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:55.401832 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.401810 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:55.410312 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.410276 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxsqn\" (UniqueName: \"kubernetes.io/projected/10453eb3-c597-4309-9b4d-2badea6493be-kube-api-access-dxsqn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:55.607106 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.607021 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:55.736504 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:55.736480 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5"] Jul 17 10:57:55.738478 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:57:55.738449 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10453eb3_c597_4309_9b4d_2badea6493be.slice/crio-94a23aa5fe389126ed82df3ec2af10e503b167672379f769180f7290cfa0b02a WatchSource:0}: Error finding container 94a23aa5fe389126ed82df3ec2af10e503b167672379f769180f7290cfa0b02a: Status 404 returned error can't find the container with id 94a23aa5fe389126ed82df3ec2af10e503b167672379f769180f7290cfa0b02a Jul 17 10:57:56.225574 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:56.225519 2551 generic.go:356] "Generic (PLEG): container finished" podID="10453eb3-c597-4309-9b4d-2badea6493be" containerID="efae19aa830b4a08ebd86ce037a3f18efa1b27209cd1c4381891734142c90385" exitCode=0 Jul 17 10:57:56.225724 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:56.225594 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" event={"ID":"10453eb3-c597-4309-9b4d-2badea6493be","Type":"ContainerDied","Data":"efae19aa830b4a08ebd86ce037a3f18efa1b27209cd1c4381891734142c90385"} Jul 17 10:57:56.225724 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:56.225653 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" event={"ID":"10453eb3-c597-4309-9b4d-2badea6493be","Type":"ContainerStarted","Data":"94a23aa5fe389126ed82df3ec2af10e503b167672379f769180f7290cfa0b02a"} Jul 17 10:57:57.231424 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:57.231389 2551 generic.go:356] "Generic (PLEG): container finished" podID="10453eb3-c597-4309-9b4d-2badea6493be" containerID="60e8481a8fa56bfb44e021c6cbea858687d0247dbdce83247055c175c280aaa7" exitCode=0 Jul 17 10:57:57.231805 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:57.231459 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" event={"ID":"10453eb3-c597-4309-9b4d-2badea6493be","Type":"ContainerDied","Data":"60e8481a8fa56bfb44e021c6cbea858687d0247dbdce83247055c175c280aaa7"} Jul 17 10:57:58.238175 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:58.238140 2551 generic.go:356] "Generic (PLEG): container finished" podID="10453eb3-c597-4309-9b4d-2badea6493be" containerID="d4c0a0cf615d7555ccaca2d054cb462a94634a402c392b9634a221f402bc77f4" exitCode=0 Jul 17 10:57:58.238528 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:58.238189 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" event={"ID":"10453eb3-c597-4309-9b4d-2badea6493be","Type":"ContainerDied","Data":"d4c0a0cf615d7555ccaca2d054cb462a94634a402c392b9634a221f402bc77f4"} Jul 17 10:57:59.368790 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:59.368756 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:57:59.433509 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:59.433481 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-util\") pod \"10453eb3-c597-4309-9b4d-2badea6493be\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " Jul 17 10:57:59.433694 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:59.433530 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-bundle\") pod \"10453eb3-c597-4309-9b4d-2badea6493be\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " Jul 17 10:57:59.433694 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:59.433588 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxsqn\" (UniqueName: \"kubernetes.io/projected/10453eb3-c597-4309-9b4d-2badea6493be-kube-api-access-dxsqn\") pod \"10453eb3-c597-4309-9b4d-2badea6493be\" (UID: \"10453eb3-c597-4309-9b4d-2badea6493be\") " Jul 17 10:57:59.434410 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:59.434375 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-bundle" (OuterVolumeSpecName: "bundle") pod "10453eb3-c597-4309-9b4d-2badea6493be" (UID: "10453eb3-c597-4309-9b4d-2badea6493be"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:57:59.435684 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:59.435638 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10453eb3-c597-4309-9b4d-2badea6493be-kube-api-access-dxsqn" (OuterVolumeSpecName: "kube-api-access-dxsqn") pod "10453eb3-c597-4309-9b4d-2badea6493be" (UID: "10453eb3-c597-4309-9b4d-2badea6493be"). InnerVolumeSpecName "kube-api-access-dxsqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:57:59.439359 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:59.439322 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-util" (OuterVolumeSpecName: "util") pod "10453eb3-c597-4309-9b4d-2badea6493be" (UID: "10453eb3-c597-4309-9b4d-2badea6493be"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:57:59.534238 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:59.534210 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:59.534238 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:59.534238 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxsqn\" (UniqueName: \"kubernetes.io/projected/10453eb3-c597-4309-9b4d-2badea6493be-kube-api-access-dxsqn\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:57:59.534409 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:57:59.534249 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10453eb3-c597-4309-9b4d-2badea6493be-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:58:00.250088 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:00.250061 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" Jul 17 10:58:00.250088 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:00.250067 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrcxf5" event={"ID":"10453eb3-c597-4309-9b4d-2badea6493be","Type":"ContainerDied","Data":"94a23aa5fe389126ed82df3ec2af10e503b167672379f769180f7290cfa0b02a"} Jul 17 10:58:00.250336 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:00.250107 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94a23aa5fe389126ed82df3ec2af10e503b167672379f769180f7290cfa0b02a" Jul 17 10:58:08.586690 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.586654 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx"] Jul 17 10:58:08.587086 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.586966 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10453eb3-c597-4309-9b4d-2badea6493be" containerName="util" Jul 17 10:58:08.587086 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.586978 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="10453eb3-c597-4309-9b4d-2badea6493be" containerName="util" Jul 17 10:58:08.587086 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.586997 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10453eb3-c597-4309-9b4d-2badea6493be" containerName="pull" Jul 17 10:58:08.587086 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.587003 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="10453eb3-c597-4309-9b4d-2badea6493be" containerName="pull" Jul 17 10:58:08.587086 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.587015 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10453eb3-c597-4309-9b4d-2badea6493be" containerName="extract" Jul 17 10:58:08.587086 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.587023 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="10453eb3-c597-4309-9b4d-2badea6493be" containerName="extract" Jul 17 10:58:08.587086 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.587079 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="10453eb3-c597-4309-9b4d-2badea6493be" containerName="extract" Jul 17 10:58:08.592935 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.592915 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.598244 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.598221 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 17 10:58:08.598640 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.598619 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 17 10:58:08.598780 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.598764 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 17 10:58:08.598832 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.598808 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 17 10:58:08.600723 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.600706 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zjf47\"" Jul 17 10:58:08.614522 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.614495 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx"] Jul 17 10:58:08.709040 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.709006 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/391cc4cd-934d-415c-b05f-f9a55d622937-webhook-cert\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.709040 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.709042 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/391cc4cd-934d-415c-b05f-f9a55d622937-manifests\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.709240 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.709079 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/391cc4cd-934d-415c-b05f-f9a55d622937-tmp\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.709240 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.709147 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvg8f\" (UniqueName: \"kubernetes.io/projected/391cc4cd-934d-415c-b05f-f9a55d622937-kube-api-access-jvg8f\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.709240 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.709182 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/391cc4cd-934d-415c-b05f-f9a55d622937-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.809862 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.809826 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/391cc4cd-934d-415c-b05f-f9a55d622937-tmp\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.810043 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.809889 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvg8f\" (UniqueName: \"kubernetes.io/projected/391cc4cd-934d-415c-b05f-f9a55d622937-kube-api-access-jvg8f\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.810043 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.809940 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/391cc4cd-934d-415c-b05f-f9a55d622937-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.810043 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.809967 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/391cc4cd-934d-415c-b05f-f9a55d622937-webhook-cert\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.810043 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.809993 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/391cc4cd-934d-415c-b05f-f9a55d622937-manifests\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.810397 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.810368 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/391cc4cd-934d-415c-b05f-f9a55d622937-manifests\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.812315 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.812283 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/391cc4cd-934d-415c-b05f-f9a55d622937-tmp\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.812440 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.812414 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/391cc4cd-934d-415c-b05f-f9a55d622937-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.812496 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.812464 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/391cc4cd-934d-415c-b05f-f9a55d622937-webhook-cert\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.826025 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.825999 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvg8f\" (UniqueName: \"kubernetes.io/projected/391cc4cd-934d-415c-b05f-f9a55d622937-kube-api-access-jvg8f\") pod \"opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx\" (UID: \"391cc4cd-934d-415c-b05f-f9a55d622937\") " pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:08.902736 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:08.902648 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:09.058014 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:09.057988 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx"] Jul 17 10:58:09.060313 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:58:09.060285 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod391cc4cd_934d_415c_b05f_f9a55d622937.slice/crio-0bb7a7d514896477669bf12138dd2a2406606a51c5bfd0e1c5c36cf197ad4e95 WatchSource:0}: Error finding container 0bb7a7d514896477669bf12138dd2a2406606a51c5bfd0e1c5c36cf197ad4e95: Status 404 returned error can't find the container with id 0bb7a7d514896477669bf12138dd2a2406606a51c5bfd0e1c5c36cf197ad4e95 Jul 17 10:58:09.293518 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:09.293481 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" event={"ID":"391cc4cd-934d-415c-b05f-f9a55d622937","Type":"ContainerStarted","Data":"0bb7a7d514896477669bf12138dd2a2406606a51c5bfd0e1c5c36cf197ad4e95"} Jul 17 10:58:13.314965 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:13.314925 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" event={"ID":"391cc4cd-934d-415c-b05f-f9a55d622937","Type":"ContainerStarted","Data":"a603e5ae28d88a3889adbda8350b07c655ac21c04cc4d14f4eb99c7b2abea78c"} Jul 17 10:58:14.321602 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:14.321565 2551 generic.go:356] "Generic (PLEG): container finished" podID="391cc4cd-934d-415c-b05f-f9a55d622937" containerID="a603e5ae28d88a3889adbda8350b07c655ac21c04cc4d14f4eb99c7b2abea78c" exitCode=0 Jul 17 10:58:14.322000 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:14.321648 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" event={"ID":"391cc4cd-934d-415c-b05f-f9a55d622937","Type":"ContainerDied","Data":"a603e5ae28d88a3889adbda8350b07c655ac21c04cc4d14f4eb99c7b2abea78c"} Jul 17 10:58:15.327931 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:15.327889 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" event={"ID":"391cc4cd-934d-415c-b05f-f9a55d622937","Type":"ContainerStarted","Data":"158bdd6e889c0104c1ddc187608f7cabd0cb4ac8d8d6505aa4f058a571e3d73e"} Jul 17 10:58:15.328359 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:15.327980 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:15.349676 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:15.349625 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" podStartSLOduration=2.707142146 podStartE2EDuration="7.349610521s" totalImagesPullingTime="3.63483233s" totalInitContainerRuntime="1.007636045s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:58:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:58:15.348168674 +0000 UTC m=+359.021684338" watchObservedRunningTime="2026-07-17 10:58:15.349610521 +0000 UTC m=+359.023126173" Jul 17 10:58:26.335313 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:26.335279 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx" Jul 17 10:58:28.543825 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.543789 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn"] Jul 17 10:58:28.553255 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.553233 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:28.556784 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.556751 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 17 10:58:28.557167 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.557144 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 17 10:58:28.557298 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.557199 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 17 10:58:28.557479 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.557465 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 17 10:58:28.580812 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.580776 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn"] Jul 17 10:58:28.686614 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.686580 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2c9ec8f-63ac-49a3-8b45-367c377cf692-tmp\") pod \"kube-auth-proxy-5445b9658d-7n5tn\" (UID: \"c2c9ec8f-63ac-49a3-8b45-367c377cf692\") " pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:28.686804 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.686640 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c9ec8f-63ac-49a3-8b45-367c377cf692-tls-certs\") pod \"kube-auth-proxy-5445b9658d-7n5tn\" (UID: \"c2c9ec8f-63ac-49a3-8b45-367c377cf692\") " pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:28.686804 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.686717 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-684tm\" (UniqueName: \"kubernetes.io/projected/c2c9ec8f-63ac-49a3-8b45-367c377cf692-kube-api-access-684tm\") pod \"kube-auth-proxy-5445b9658d-7n5tn\" (UID: \"c2c9ec8f-63ac-49a3-8b45-367c377cf692\") " pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:28.787692 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.787659 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2c9ec8f-63ac-49a3-8b45-367c377cf692-tmp\") pod \"kube-auth-proxy-5445b9658d-7n5tn\" (UID: \"c2c9ec8f-63ac-49a3-8b45-367c377cf692\") " pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:28.787692 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.787707 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c9ec8f-63ac-49a3-8b45-367c377cf692-tls-certs\") pod \"kube-auth-proxy-5445b9658d-7n5tn\" (UID: \"c2c9ec8f-63ac-49a3-8b45-367c377cf692\") " pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:28.787897 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.787739 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-684tm\" (UniqueName: \"kubernetes.io/projected/c2c9ec8f-63ac-49a3-8b45-367c377cf692-kube-api-access-684tm\") pod \"kube-auth-proxy-5445b9658d-7n5tn\" (UID: \"c2c9ec8f-63ac-49a3-8b45-367c377cf692\") " pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:28.787897 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:58:28.787848 2551 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 17 10:58:28.787962 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:58:28.787931 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2c9ec8f-63ac-49a3-8b45-367c377cf692-tls-certs podName:c2c9ec8f-63ac-49a3-8b45-367c377cf692 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:29.287908406 +0000 UTC m=+372.961424049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c2c9ec8f-63ac-49a3-8b45-367c377cf692-tls-certs") pod "kube-auth-proxy-5445b9658d-7n5tn" (UID: "c2c9ec8f-63ac-49a3-8b45-367c377cf692") : secret "kube-auth-proxy-tls" not found Jul 17 10:58:28.789894 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.789868 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2c9ec8f-63ac-49a3-8b45-367c377cf692-tmp\") pod \"kube-auth-proxy-5445b9658d-7n5tn\" (UID: \"c2c9ec8f-63ac-49a3-8b45-367c377cf692\") " pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:28.811039 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:28.810967 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-684tm\" (UniqueName: \"kubernetes.io/projected/c2c9ec8f-63ac-49a3-8b45-367c377cf692-kube-api-access-684tm\") pod \"kube-auth-proxy-5445b9658d-7n5tn\" (UID: \"c2c9ec8f-63ac-49a3-8b45-367c377cf692\") " pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:29.291573 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:29.291513 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c9ec8f-63ac-49a3-8b45-367c377cf692-tls-certs\") pod \"kube-auth-proxy-5445b9658d-7n5tn\" (UID: \"c2c9ec8f-63ac-49a3-8b45-367c377cf692\") " pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:29.294048 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:29.294022 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c9ec8f-63ac-49a3-8b45-367c377cf692-tls-certs\") pod \"kube-auth-proxy-5445b9658d-7n5tn\" (UID: \"c2c9ec8f-63ac-49a3-8b45-367c377cf692\") " pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:29.463485 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:29.463440 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" Jul 17 10:58:29.613721 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:29.613689 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn"] Jul 17 10:58:29.615734 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:58:29.615708 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c9ec8f_63ac_49a3_8b45_367c377cf692.slice/crio-150fa54d55573659cc6ecaa80e0be21b326415e554e1d099e43813f2bdf59ee8 WatchSource:0}: Error finding container 150fa54d55573659cc6ecaa80e0be21b326415e554e1d099e43813f2bdf59ee8: Status 404 returned error can't find the container with id 150fa54d55573659cc6ecaa80e0be21b326415e554e1d099e43813f2bdf59ee8 Jul 17 10:58:29.886865 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:29.886785 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fwnbx"] Jul 17 10:58:29.891339 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:29.891323 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" Jul 17 10:58:29.895971 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:29.895951 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 17 10:58:29.896078 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:29.896003 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-dpt2l\"" Jul 17 10:58:29.910185 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:29.910158 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fwnbx"] Jul 17 10:58:29.997245 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:29.997210 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d-tls-certs\") pod \"model-serving-api-54d89bd5fc-fwnbx\" (UID: \"d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" Jul 17 10:58:29.997416 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:29.997305 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp5c5\" (UniqueName: \"kubernetes.io/projected/d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d-kube-api-access-cp5c5\") pod \"model-serving-api-54d89bd5fc-fwnbx\" (UID: \"d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" Jul 17 10:58:30.097955 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:30.097924 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp5c5\" (UniqueName: \"kubernetes.io/projected/d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d-kube-api-access-cp5c5\") pod \"model-serving-api-54d89bd5fc-fwnbx\" (UID: \"d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" Jul 17 10:58:30.098136 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:30.097968 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d-tls-certs\") pod \"model-serving-api-54d89bd5fc-fwnbx\" (UID: \"d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" Jul 17 10:58:30.100441 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:30.100412 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d-tls-certs\") pod \"model-serving-api-54d89bd5fc-fwnbx\" (UID: \"d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" Jul 17 10:58:30.108001 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:30.107978 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp5c5\" (UniqueName: \"kubernetes.io/projected/d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d-kube-api-access-cp5c5\") pod \"model-serving-api-54d89bd5fc-fwnbx\" (UID: \"d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" Jul 17 10:58:30.204412 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:30.204292 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" Jul 17 10:58:30.333780 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:30.333749 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fwnbx"] Jul 17 10:58:30.335781 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:58:30.335752 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd67fb8ad_a6ca_4d7e_a73e_a2862cb35e0d.slice/crio-9e4cbb52d8d4d068ff76b49ab6282e11cb822c2947876d6d4ff9557d6cc6cceb WatchSource:0}: Error finding container 9e4cbb52d8d4d068ff76b49ab6282e11cb822c2947876d6d4ff9557d6cc6cceb: Status 404 returned error can't find the container with id 9e4cbb52d8d4d068ff76b49ab6282e11cb822c2947876d6d4ff9557d6cc6cceb Jul 17 10:58:30.394419 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:30.394377 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" event={"ID":"d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d","Type":"ContainerStarted","Data":"9e4cbb52d8d4d068ff76b49ab6282e11cb822c2947876d6d4ff9557d6cc6cceb"} Jul 17 10:58:30.395529 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:30.395501 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" event={"ID":"c2c9ec8f-63ac-49a3-8b45-367c377cf692","Type":"ContainerStarted","Data":"150fa54d55573659cc6ecaa80e0be21b326415e554e1d099e43813f2bdf59ee8"} Jul 17 10:58:33.412382 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:33.412343 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" event={"ID":"d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d","Type":"ContainerStarted","Data":"35eecdd834d97e7182bc6c2f0a6963d379d4c2b5e57890abed0f59080981c1b3"} Jul 17 10:58:33.412869 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:33.412516 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" Jul 17 10:58:33.413879 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:33.413858 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" event={"ID":"c2c9ec8f-63ac-49a3-8b45-367c377cf692","Type":"ContainerStarted","Data":"913fa1934bdc7587314db0ff7ea4ffffeed8286c1e0725586e67d956f05da8dd"} Jul 17 10:58:33.446312 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:33.446268 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" podStartSLOduration=2.341982712 podStartE2EDuration="4.446255819s" totalImagesPullingTime="2.104273107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:58:33.445100518 +0000 UTC m=+377.118616183" watchObservedRunningTime="2026-07-17 10:58:33.446255819 +0000 UTC m=+377.119771471" Jul 17 10:58:33.462258 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:33.462212 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5445b9658d-7n5tn" podStartSLOduration=2.631642336 podStartE2EDuration="5.462199239s" totalImagesPullingTime="2.830556903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:58:33.461639193 +0000 UTC m=+377.135154844" watchObservedRunningTime="2026-07-17 10:58:33.462199239 +0000 UTC m=+377.135714901" Jul 17 10:58:36.811833 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.811797 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg"] Jul 17 10:58:36.815368 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.815346 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:36.819697 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.819664 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 17 10:58:36.819839 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.819801 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 17 10:58:36.819918 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.819810 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-r47p7\"" Jul 17 10:58:36.819918 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.819895 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 17 10:58:36.820091 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.820075 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 17 10:58:36.834673 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.834645 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg"] Jul 17 10:58:36.962245 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.962197 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:36.962443 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.962296 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:36.962443 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.962335 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zklxk\" (UniqueName: \"kubernetes.io/projected/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-kube-api-access-zklxk\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:36.962443 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.962381 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:36.962576 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.962499 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:36.962618 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.962584 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:36.962653 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:36.962617 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.063382 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.063264 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.063382 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.063328 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.063629 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.063394 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.063629 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.063472 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.063629 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.063498 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zklxk\" (UniqueName: \"kubernetes.io/projected/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-kube-api-access-zklxk\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.063629 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.063537 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.063839 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.063762 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.064149 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.064120 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.066598 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.066570 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.066598 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.066585 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.066778 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.066600 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.067161 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.067140 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.081281 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.081254 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zklxk\" (UniqueName: \"kubernetes.io/projected/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-kube-api-access-zklxk\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.082602 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.082578 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6eedd6f-c67e-41c4-b5b6-36c25f71b139-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8xcdg\" (UID: \"c6eedd6f-c67e-41c4-b5b6-36c25f71b139\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.124989 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.124959 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:37.294987 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.294958 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg"] Jul 17 10:58:37.295920 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:58:37.295888 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6eedd6f_c67e_41c4_b5b6_36c25f71b139.slice/crio-5803156dfbd42d23a619b02e89ab2f0655295d509b71a06e0291f42ea1b2663d WatchSource:0}: Error finding container 5803156dfbd42d23a619b02e89ab2f0655295d509b71a06e0291f42ea1b2663d: Status 404 returned error can't find the container with id 5803156dfbd42d23a619b02e89ab2f0655295d509b71a06e0291f42ea1b2663d Jul 17 10:58:37.433650 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:37.433538 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" event={"ID":"c6eedd6f-c67e-41c4-b5b6-36c25f71b139","Type":"ContainerStarted","Data":"5803156dfbd42d23a619b02e89ab2f0655295d509b71a06e0291f42ea1b2663d"} Jul 17 10:58:39.751693 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:39.751642 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 10:58:39.752057 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:39.751748 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 10:58:40.055840 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.055795 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq"] Jul 17 10:58:40.059279 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.059259 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" Jul 17 10:58:40.062968 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.062938 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv"] Jul 17 10:58:40.066429 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.066193 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-dzfnz\"" Jul 17 10:58:40.066429 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.066343 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 17 10:58:40.072113 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.067117 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" Jul 17 10:58:40.076142 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.076121 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 17 10:58:40.076235 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.076190 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-87k4t\"" Jul 17 10:58:40.089512 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.089487 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq"] Jul 17 10:58:40.091833 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.091811 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv"] Jul 17 10:58:40.192461 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.192416 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnmv7\" (UniqueName: \"kubernetes.io/projected/78425956-706c-4299-87fe-670db5f71eb9-kube-api-access-dnmv7\") pod \"llmisvc-controller-manager-777c6bd7d5-nsvgv\" (UID: \"78425956-706c-4299-87fe-670db5f71eb9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" Jul 17 10:58:40.192701 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.192520 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53ca2f01-0fa9-4a5f-9e1a-12303c1c0322-cert\") pod \"kserve-controller-manager-d6f4b65b8-qjlhq\" (UID: \"53ca2f01-0fa9-4a5f-9e1a-12303c1c0322\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" Jul 17 10:58:40.192701 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.192624 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj7c8\" (UniqueName: \"kubernetes.io/projected/53ca2f01-0fa9-4a5f-9e1a-12303c1c0322-kube-api-access-hj7c8\") pod \"kserve-controller-manager-d6f4b65b8-qjlhq\" (UID: \"53ca2f01-0fa9-4a5f-9e1a-12303c1c0322\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" Jul 17 10:58:40.192701 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.192653 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78425956-706c-4299-87fe-670db5f71eb9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nsvgv\" (UID: \"78425956-706c-4299-87fe-670db5f71eb9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" Jul 17 10:58:40.294012 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.293970 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53ca2f01-0fa9-4a5f-9e1a-12303c1c0322-cert\") pod \"kserve-controller-manager-d6f4b65b8-qjlhq\" (UID: \"53ca2f01-0fa9-4a5f-9e1a-12303c1c0322\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" Jul 17 10:58:40.294180 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.294028 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj7c8\" (UniqueName: \"kubernetes.io/projected/53ca2f01-0fa9-4a5f-9e1a-12303c1c0322-kube-api-access-hj7c8\") pod \"kserve-controller-manager-d6f4b65b8-qjlhq\" (UID: \"53ca2f01-0fa9-4a5f-9e1a-12303c1c0322\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" Jul 17 10:58:40.294180 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.294056 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78425956-706c-4299-87fe-670db5f71eb9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nsvgv\" (UID: \"78425956-706c-4299-87fe-670db5f71eb9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" Jul 17 10:58:40.294180 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.294125 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnmv7\" (UniqueName: \"kubernetes.io/projected/78425956-706c-4299-87fe-670db5f71eb9-kube-api-access-dnmv7\") pod \"llmisvc-controller-manager-777c6bd7d5-nsvgv\" (UID: \"78425956-706c-4299-87fe-670db5f71eb9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" Jul 17 10:58:40.294180 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:58:40.294134 2551 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 17 10:58:40.294327 ip-10-0-143-42 kubenswrapper[2551]: E0717 10:58:40.294214 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53ca2f01-0fa9-4a5f-9e1a-12303c1c0322-cert podName:53ca2f01-0fa9-4a5f-9e1a-12303c1c0322 nodeName:}" failed. No retries permitted until 2026-07-17 10:58:40.79419054 +0000 UTC m=+384.467706180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/53ca2f01-0fa9-4a5f-9e1a-12303c1c0322-cert") pod "kserve-controller-manager-d6f4b65b8-qjlhq" (UID: "53ca2f01-0fa9-4a5f-9e1a-12303c1c0322") : secret "kserve-webhook-server-cert" not found Jul 17 10:58:40.296606 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.296578 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78425956-706c-4299-87fe-670db5f71eb9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nsvgv\" (UID: \"78425956-706c-4299-87fe-670db5f71eb9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" Jul 17 10:58:40.315041 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.314962 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnmv7\" (UniqueName: \"kubernetes.io/projected/78425956-706c-4299-87fe-670db5f71eb9-kube-api-access-dnmv7\") pod \"llmisvc-controller-manager-777c6bd7d5-nsvgv\" (UID: \"78425956-706c-4299-87fe-670db5f71eb9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" Jul 17 10:58:40.315041 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.314999 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj7c8\" (UniqueName: \"kubernetes.io/projected/53ca2f01-0fa9-4a5f-9e1a-12303c1c0322-kube-api-access-hj7c8\") pod \"kserve-controller-manager-d6f4b65b8-qjlhq\" (UID: \"53ca2f01-0fa9-4a5f-9e1a-12303c1c0322\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" Jul 17 10:58:40.384257 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.384151 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" Jul 17 10:58:40.454323 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.452956 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" event={"ID":"c6eedd6f-c67e-41c4-b5b6-36c25f71b139","Type":"ContainerStarted","Data":"af53c92b1fa3f1a13b785ce0ee489ecb4a81a97fcc25316704466b5db8a65aec"} Jul 17 10:58:40.454323 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.453746 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:40.455192 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.455029 2551 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-8xcdg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 10:58:40.455192 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.455103 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" podUID="c6eedd6f-c67e-41c4-b5b6-36c25f71b139" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 10:58:40.535602 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.535526 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" podStartSLOduration=2.082268786 podStartE2EDuration="4.535506995s" totalImagesPullingTime="2.453238209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:58:40.533903192 +0000 UTC m=+384.207418845" watchObservedRunningTime="2026-07-17 10:58:40.535506995 +0000 UTC m=+384.209022646" Jul 17 10:58:40.575967 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.575762 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv"] Jul 17 10:58:40.578709 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:58:40.578682 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod78425956_706c_4299_87fe_670db5f71eb9.slice/crio-86679e14833800a3e76bdbfc219578eebc8bd35679b74466019168e88814e475 WatchSource:0}: Error finding container 86679e14833800a3e76bdbfc219578eebc8bd35679b74466019168e88814e475: Status 404 returned error can't find the container with id 86679e14833800a3e76bdbfc219578eebc8bd35679b74466019168e88814e475 Jul 17 10:58:40.799896 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.799856 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53ca2f01-0fa9-4a5f-9e1a-12303c1c0322-cert\") pod \"kserve-controller-manager-d6f4b65b8-qjlhq\" (UID: \"53ca2f01-0fa9-4a5f-9e1a-12303c1c0322\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" Jul 17 10:58:40.802214 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.802194 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53ca2f01-0fa9-4a5f-9e1a-12303c1c0322-cert\") pod \"kserve-controller-manager-d6f4b65b8-qjlhq\" (UID: \"53ca2f01-0fa9-4a5f-9e1a-12303c1c0322\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" Jul 17 10:58:40.975901 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:40.975817 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" Jul 17 10:58:41.129090 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:41.129055 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq"] Jul 17 10:58:41.132944 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:58:41.132914 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ca2f01_0fa9_4a5f_9e1a_12303c1c0322.slice/crio-f6401c2493217898a67dcc7aa5955cf3c52b36e5e949fc2c6c2daebcc83944c5 WatchSource:0}: Error finding container f6401c2493217898a67dcc7aa5955cf3c52b36e5e949fc2c6c2daebcc83944c5: Status 404 returned error can't find the container with id f6401c2493217898a67dcc7aa5955cf3c52b36e5e949fc2c6c2daebcc83944c5 Jul 17 10:58:41.460080 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:41.460041 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" event={"ID":"53ca2f01-0fa9-4a5f-9e1a-12303c1c0322","Type":"ContainerStarted","Data":"f6401c2493217898a67dcc7aa5955cf3c52b36e5e949fc2c6c2daebcc83944c5"} Jul 17 10:58:41.461191 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:41.461164 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" event={"ID":"78425956-706c-4299-87fe-670db5f71eb9","Type":"ContainerStarted","Data":"86679e14833800a3e76bdbfc219578eebc8bd35679b74466019168e88814e475"} Jul 17 10:58:41.461915 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:41.461889 2551 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-8xcdg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 10:58:41.462007 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:41.461938 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" podUID="c6eedd6f-c67e-41c4-b5b6-36c25f71b139" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 10:58:42.468176 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:42.468147 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8xcdg" Jul 17 10:58:44.422694 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:44.422614 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fwnbx" Jul 17 10:58:44.481919 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:44.481822 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" event={"ID":"53ca2f01-0fa9-4a5f-9e1a-12303c1c0322","Type":"ContainerStarted","Data":"9834fe458a0fb8e3ab6b8bf96ab1eff7ee0880b1838d6a788c09a6593683206b"} Jul 17 10:58:44.481919 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:44.481921 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" Jul 17 10:58:44.483532 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:44.483504 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" event={"ID":"78425956-706c-4299-87fe-670db5f71eb9","Type":"ContainerStarted","Data":"d3ec3fa9e770694d145000024447599bfdd404b4aabe3e234c5b9d0f3118cc2d"} Jul 17 10:58:44.483653 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:44.483583 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" Jul 17 10:58:44.507537 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:44.507486 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" podStartSLOduration=1.5341832229999999 podStartE2EDuration="4.507471698s" totalImagesPullingTime="2.973288475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:58:44.506024985 +0000 UTC m=+388.179540638" watchObservedRunningTime="2026-07-17 10:58:44.507471698 +0000 UTC m=+388.180987349" Jul 17 10:58:44.531209 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:58:44.531157 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" podStartSLOduration=0.99829981 podStartE2EDuration="4.531141522s" totalImagesPullingTime="3.532841712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:58:44.529570203 +0000 UTC m=+388.203085845" watchObservedRunningTime="2026-07-17 10:58:44.531141522 +0000 UTC m=+388.204657198" Jul 17 10:59:15.490225 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:15.490193 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nsvgv" Jul 17 10:59:15.493076 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:15.493058 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qjlhq" Jul 17 10:59:45.056976 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.056896 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z"] Jul 17 10:59:45.060526 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.060505 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:45.063320 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.063299 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 17 10:59:45.064321 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.064301 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 17 10:59:45.064384 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.064302 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gl6gr\"" Jul 17 10:59:45.071215 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.071190 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z"] Jul 17 10:59:45.237227 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.237185 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:45.237227 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.237223 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:45.237440 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.237253 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-699w7\" (UniqueName: \"kubernetes.io/projected/fc0649e1-69e8-46ae-8bf1-ae5442e11997-kube-api-access-699w7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:45.338761 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.338676 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:45.338761 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.338715 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:45.338761 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.338738 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699w7\" (UniqueName: \"kubernetes.io/projected/fc0649e1-69e8-46ae-8bf1-ae5442e11997-kube-api-access-699w7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:45.339163 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.339140 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:45.339201 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.339150 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:45.347185 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.347153 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-699w7\" (UniqueName: \"kubernetes.io/projected/fc0649e1-69e8-46ae-8bf1-ae5442e11997-kube-api-access-699w7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:45.368926 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.368899 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:45.504599 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.504563 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z"] Jul 17 10:59:45.506404 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:59:45.506379 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0649e1_69e8_46ae_8bf1_ae5442e11997.slice/crio-7eabc91f6d353de376b022d3ebd49ee7eb0e5f92d072b95dbfd960fc54b13401 WatchSource:0}: Error finding container 7eabc91f6d353de376b022d3ebd49ee7eb0e5f92d072b95dbfd960fc54b13401: Status 404 returned error can't find the container with id 7eabc91f6d353de376b022d3ebd49ee7eb0e5f92d072b95dbfd960fc54b13401 Jul 17 10:59:45.656722 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.656690 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg"] Jul 17 10:59:45.660071 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.660055 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:45.670621 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.670592 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg"] Jul 17 10:59:45.769701 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.769670 2551 generic.go:356] "Generic (PLEG): container finished" podID="fc0649e1-69e8-46ae-8bf1-ae5442e11997" containerID="a5761eed08558596bccbfe4da5e4bcd0ba3898246cf9572c4351ade093498e37" exitCode=0 Jul 17 10:59:45.769908 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.769717 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" event={"ID":"fc0649e1-69e8-46ae-8bf1-ae5442e11997","Type":"ContainerDied","Data":"a5761eed08558596bccbfe4da5e4bcd0ba3898246cf9572c4351ade093498e37"} Jul 17 10:59:45.769908 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.769751 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" event={"ID":"fc0649e1-69e8-46ae-8bf1-ae5442e11997","Type":"ContainerStarted","Data":"7eabc91f6d353de376b022d3ebd49ee7eb0e5f92d072b95dbfd960fc54b13401"} Jul 17 10:59:45.843299 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.843262 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kwn6\" (UniqueName: \"kubernetes.io/projected/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-kube-api-access-6kwn6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:45.843484 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.843331 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:45.843484 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.843360 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:45.944327 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.944228 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kwn6\" (UniqueName: \"kubernetes.io/projected/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-kube-api-access-6kwn6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:45.944497 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.944442 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:45.944593 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.944498 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:45.944844 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.944824 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:45.944908 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.944864 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:45.953320 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.953296 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kwn6\" (UniqueName: \"kubernetes.io/projected/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-kube-api-access-6kwn6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:45.973623 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:45.973600 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:46.068384 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.068354 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz"] Jul 17 10:59:46.081527 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.080896 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:46.084571 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.084485 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz"] Jul 17 10:59:46.117338 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.117310 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg"] Jul 17 10:59:46.119105 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:59:46.119070 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1bc1939_5be6_4acc_9a77_ff0c82e9e9f9.slice/crio-d0404192d607094b7d04891120461cba5f6e99b09cde016dcc1a37f471de52e2 WatchSource:0}: Error finding container d0404192d607094b7d04891120461cba5f6e99b09cde016dcc1a37f471de52e2: Status 404 returned error can't find the container with id d0404192d607094b7d04891120461cba5f6e99b09cde016dcc1a37f471de52e2 Jul 17 10:59:46.247849 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.247814 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:46.248023 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.247859 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:46.248023 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.247932 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdlrx\" (UniqueName: \"kubernetes.io/projected/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-kube-api-access-fdlrx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:46.348481 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.348442 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdlrx\" (UniqueName: \"kubernetes.io/projected/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-kube-api-access-fdlrx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:46.348681 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.348522 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:46.348681 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.348558 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:46.348915 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.348897 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:46.348974 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.348950 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:46.358307 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.358277 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdlrx\" (UniqueName: \"kubernetes.io/projected/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-kube-api-access-fdlrx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:46.391875 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.391845 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:46.469230 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.469140 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj"] Jul 17 10:59:46.473101 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.473078 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:46.483539 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.483516 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj"] Jul 17 10:59:46.649970 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.649932 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:46.650132 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.649992 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:46.650132 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.650011 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dznt8\" (UniqueName: \"kubernetes.io/projected/34081b88-e909-4671-a7a4-93ced58d7c3a-kube-api-access-dznt8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:46.659693 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.659669 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz"] Jul 17 10:59:46.661610 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:59:46.661579 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c6d33f7_0a85_4a6b_8723_6d1aefdc9689.slice/crio-398b6d8595910418d973c0c6475e73ed8da19e5035b72038f2dd1117dd006928 WatchSource:0}: Error finding container 398b6d8595910418d973c0c6475e73ed8da19e5035b72038f2dd1117dd006928: Status 404 returned error can't find the container with id 398b6d8595910418d973c0c6475e73ed8da19e5035b72038f2dd1117dd006928 Jul 17 10:59:46.751034 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.751006 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:46.751137 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.751064 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:46.751137 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.751090 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dznt8\" (UniqueName: \"kubernetes.io/projected/34081b88-e909-4671-a7a4-93ced58d7c3a-kube-api-access-dznt8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:46.751414 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.751391 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:46.751491 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.751447 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:46.760634 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.760611 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dznt8\" (UniqueName: \"kubernetes.io/projected/34081b88-e909-4671-a7a4-93ced58d7c3a-kube-api-access-dznt8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:46.776217 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.776185 2551 generic.go:356] "Generic (PLEG): container finished" podID="a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" containerID="da4e0685f5660e83e9095a5c82cc5117928c0519251405046b8954e0d993aaa3" exitCode=0 Jul 17 10:59:46.776341 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.776272 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" event={"ID":"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9","Type":"ContainerDied","Data":"da4e0685f5660e83e9095a5c82cc5117928c0519251405046b8954e0d993aaa3"} Jul 17 10:59:46.776341 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.776322 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" event={"ID":"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9","Type":"ContainerStarted","Data":"d0404192d607094b7d04891120461cba5f6e99b09cde016dcc1a37f471de52e2"} Jul 17 10:59:46.778030 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.778003 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" event={"ID":"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689","Type":"ContainerStarted","Data":"e0d89e20d830e5a9bd992805b4226639f3c7746d18f339e8d14a6cb0d09dbaec"} Jul 17 10:59:46.778151 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.778049 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" event={"ID":"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689","Type":"ContainerStarted","Data":"398b6d8595910418d973c0c6475e73ed8da19e5035b72038f2dd1117dd006928"} Jul 17 10:59:46.780025 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.780005 2551 generic.go:356] "Generic (PLEG): container finished" podID="fc0649e1-69e8-46ae-8bf1-ae5442e11997" containerID="0819d823af44885284bc1cc43c2ffd3fa452c525f36e3e43d62d8ce276cad244" exitCode=0 Jul 17 10:59:46.780107 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.780044 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" event={"ID":"fc0649e1-69e8-46ae-8bf1-ae5442e11997","Type":"ContainerDied","Data":"0819d823af44885284bc1cc43c2ffd3fa452c525f36e3e43d62d8ce276cad244"} Jul 17 10:59:46.783950 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.783935 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:46.936880 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:46.936855 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj"] Jul 17 10:59:46.939105 ip-10-0-143-42 kubenswrapper[2551]: W0717 10:59:46.939075 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34081b88_e909_4671_a7a4_93ced58d7c3a.slice/crio-123972068ffc4d5183b75ad85912ee614ec5f8a4d0a655787a21997d8a7c9a8c WatchSource:0}: Error finding container 123972068ffc4d5183b75ad85912ee614ec5f8a4d0a655787a21997d8a7c9a8c: Status 404 returned error can't find the container with id 123972068ffc4d5183b75ad85912ee614ec5f8a4d0a655787a21997d8a7c9a8c Jul 17 10:59:47.786341 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:47.786301 2551 generic.go:356] "Generic (PLEG): container finished" podID="34081b88-e909-4671-a7a4-93ced58d7c3a" containerID="0e8beaf5c3e3e8a6bb500facfbecda535b8f46791490f89f1fd0aa6e2017627a" exitCode=0 Jul 17 10:59:47.786818 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:47.786383 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" event={"ID":"34081b88-e909-4671-a7a4-93ced58d7c3a","Type":"ContainerDied","Data":"0e8beaf5c3e3e8a6bb500facfbecda535b8f46791490f89f1fd0aa6e2017627a"} Jul 17 10:59:47.786818 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:47.786441 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" event={"ID":"34081b88-e909-4671-a7a4-93ced58d7c3a","Type":"ContainerStarted","Data":"123972068ffc4d5183b75ad85912ee614ec5f8a4d0a655787a21997d8a7c9a8c"} Jul 17 10:59:47.788831 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:47.788810 2551 generic.go:356] "Generic (PLEG): container finished" podID="fc0649e1-69e8-46ae-8bf1-ae5442e11997" containerID="09c2960eb165343a9712f1546b2f92694838d05aea4ca803bee4ac2a8fd57c05" exitCode=0 Jul 17 10:59:47.788922 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:47.788890 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" event={"ID":"fc0649e1-69e8-46ae-8bf1-ae5442e11997","Type":"ContainerDied","Data":"09c2960eb165343a9712f1546b2f92694838d05aea4ca803bee4ac2a8fd57c05"} Jul 17 10:59:47.790771 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:47.790751 2551 generic.go:356] "Generic (PLEG): container finished" podID="a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" containerID="f770aa6f8bebbe07fa634b29b5ea1849cc5be91236a496eed19756297af1c4b1" exitCode=0 Jul 17 10:59:47.790871 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:47.790795 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" event={"ID":"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9","Type":"ContainerDied","Data":"f770aa6f8bebbe07fa634b29b5ea1849cc5be91236a496eed19756297af1c4b1"} Jul 17 10:59:47.792370 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:47.792350 2551 generic.go:356] "Generic (PLEG): container finished" podID="5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" containerID="e0d89e20d830e5a9bd992805b4226639f3c7746d18f339e8d14a6cb0d09dbaec" exitCode=0 Jul 17 10:59:47.792474 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:47.792389 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" event={"ID":"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689","Type":"ContainerDied","Data":"e0d89e20d830e5a9bd992805b4226639f3c7746d18f339e8d14a6cb0d09dbaec"} Jul 17 10:59:48.799323 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.799284 2551 generic.go:356] "Generic (PLEG): container finished" podID="a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" containerID="3f8ab8f306a838c647e66af141302f30514bbe9e1ef490621087243b5fdb5a97" exitCode=0 Jul 17 10:59:48.799748 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.799363 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" event={"ID":"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9","Type":"ContainerDied","Data":"3f8ab8f306a838c647e66af141302f30514bbe9e1ef490621087243b5fdb5a97"} Jul 17 10:59:48.801411 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.801390 2551 generic.go:356] "Generic (PLEG): container finished" podID="5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" containerID="806dc0ef8b1dcab89d30cfb1b2413ac15142925bc4e7e8eede5bc1b33c34c733" exitCode=0 Jul 17 10:59:48.801509 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.801452 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" event={"ID":"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689","Type":"ContainerDied","Data":"806dc0ef8b1dcab89d30cfb1b2413ac15142925bc4e7e8eede5bc1b33c34c733"} Jul 17 10:59:48.803537 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.803517 2551 generic.go:356] "Generic (PLEG): container finished" podID="34081b88-e909-4671-a7a4-93ced58d7c3a" containerID="329ab0969300fbcf0eb5354286fef0a4ed7bc6ed0bb018775e933b97fd2b1ea1" exitCode=0 Jul 17 10:59:48.803655 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.803580 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" event={"ID":"34081b88-e909-4671-a7a4-93ced58d7c3a","Type":"ContainerDied","Data":"329ab0969300fbcf0eb5354286fef0a4ed7bc6ed0bb018775e933b97fd2b1ea1"} Jul 17 10:59:48.954691 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.954665 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:48.968475 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.968445 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-699w7\" (UniqueName: \"kubernetes.io/projected/fc0649e1-69e8-46ae-8bf1-ae5442e11997-kube-api-access-699w7\") pod \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " Jul 17 10:59:48.968646 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.968497 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-util\") pod \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " Jul 17 10:59:48.968703 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.968689 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-bundle\") pod \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\" (UID: \"fc0649e1-69e8-46ae-8bf1-ae5442e11997\") " Jul 17 10:59:48.969119 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.969087 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-bundle" (OuterVolumeSpecName: "bundle") pod "fc0649e1-69e8-46ae-8bf1-ae5442e11997" (UID: "fc0649e1-69e8-46ae-8bf1-ae5442e11997"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:59:48.970826 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.970795 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0649e1-69e8-46ae-8bf1-ae5442e11997-kube-api-access-699w7" (OuterVolumeSpecName: "kube-api-access-699w7") pod "fc0649e1-69e8-46ae-8bf1-ae5442e11997" (UID: "fc0649e1-69e8-46ae-8bf1-ae5442e11997"). InnerVolumeSpecName "kube-api-access-699w7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:59:48.974856 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:48.974825 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-util" (OuterVolumeSpecName: "util") pod "fc0649e1-69e8-46ae-8bf1-ae5442e11997" (UID: "fc0649e1-69e8-46ae-8bf1-ae5442e11997"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:59:49.069333 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.069303 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:49.069333 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.069330 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-699w7\" (UniqueName: \"kubernetes.io/projected/fc0649e1-69e8-46ae-8bf1-ae5442e11997-kube-api-access-699w7\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:49.069333 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.069339 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc0649e1-69e8-46ae-8bf1-ae5442e11997-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:49.811139 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.811108 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" Jul 17 10:59:49.811524 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.811137 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z" event={"ID":"fc0649e1-69e8-46ae-8bf1-ae5442e11997","Type":"ContainerDied","Data":"7eabc91f6d353de376b022d3ebd49ee7eb0e5f92d072b95dbfd960fc54b13401"} Jul 17 10:59:49.811524 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.811189 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7eabc91f6d353de376b022d3ebd49ee7eb0e5f92d072b95dbfd960fc54b13401" Jul 17 10:59:49.813176 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.813154 2551 generic.go:356] "Generic (PLEG): container finished" podID="5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" containerID="08993e5c142f38f0bd3fdc88c8848ad913c51a39aff0b7fbd41d58323966fbf0" exitCode=0 Jul 17 10:59:49.813288 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.813244 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" event={"ID":"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689","Type":"ContainerDied","Data":"08993e5c142f38f0bd3fdc88c8848ad913c51a39aff0b7fbd41d58323966fbf0"} Jul 17 10:59:49.815254 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.815231 2551 generic.go:356] "Generic (PLEG): container finished" podID="34081b88-e909-4671-a7a4-93ced58d7c3a" containerID="e41ef7f6baecc608b740da27cea727914985c81ac994bbc832ac0943c0d3e828" exitCode=0 Jul 17 10:59:49.815368 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.815319 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" event={"ID":"34081b88-e909-4671-a7a4-93ced58d7c3a","Type":"ContainerDied","Data":"e41ef7f6baecc608b740da27cea727914985c81ac994bbc832ac0943c0d3e828"} Jul 17 10:59:49.946674 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.946643 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:49.975035 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.975006 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kwn6\" (UniqueName: \"kubernetes.io/projected/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-kube-api-access-6kwn6\") pod \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " Jul 17 10:59:49.975188 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.975095 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-bundle\") pod \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " Jul 17 10:59:49.975188 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.975136 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-util\") pod \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\" (UID: \"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9\") " Jul 17 10:59:49.975862 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.975834 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-bundle" (OuterVolumeSpecName: "bundle") pod "a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" (UID: "a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:59:49.977128 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.977103 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-kube-api-access-6kwn6" (OuterVolumeSpecName: "kube-api-access-6kwn6") pod "a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" (UID: "a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9"). InnerVolumeSpecName "kube-api-access-6kwn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:59:49.980207 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:49.980181 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-util" (OuterVolumeSpecName: "util") pod "a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" (UID: "a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:59:50.076006 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.075957 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:50.076006 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.075994 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:50.076006 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.076006 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kwn6\" (UniqueName: \"kubernetes.io/projected/a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9-kube-api-access-6kwn6\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:50.823270 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.823238 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" Jul 17 10:59:50.823721 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.823268 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg" event={"ID":"a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9","Type":"ContainerDied","Data":"d0404192d607094b7d04891120461cba5f6e99b09cde016dcc1a37f471de52e2"} Jul 17 10:59:50.823721 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.823313 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0404192d607094b7d04891120461cba5f6e99b09cde016dcc1a37f471de52e2" Jul 17 10:59:50.982000 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.981975 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 10:59:50.982314 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.982296 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-bundle\") pod \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " Jul 17 10:59:50.982392 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.982364 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdlrx\" (UniqueName: \"kubernetes.io/projected/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-kube-api-access-fdlrx\") pod \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " Jul 17 10:59:50.982392 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.982389 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-util\") pod \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\" (UID: \"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689\") " Jul 17 10:59:50.982836 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.982813 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-bundle" (OuterVolumeSpecName: "bundle") pod "5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" (UID: "5c6d33f7-0a85-4a6b-8723-6d1aefdc9689"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:59:50.984232 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.984211 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:50.984332 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.984259 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-kube-api-access-fdlrx" (OuterVolumeSpecName: "kube-api-access-fdlrx") pod "5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" (UID: "5c6d33f7-0a85-4a6b-8723-6d1aefdc9689"). InnerVolumeSpecName "kube-api-access-fdlrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:59:50.988177 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:50.988134 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-util" (OuterVolumeSpecName: "util") pod "5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" (UID: "5c6d33f7-0a85-4a6b-8723-6d1aefdc9689"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:59:51.083656 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.083543 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-bundle\") pod \"34081b88-e909-4671-a7a4-93ced58d7c3a\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " Jul 17 10:59:51.083656 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.083606 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-util\") pod \"34081b88-e909-4671-a7a4-93ced58d7c3a\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " Jul 17 10:59:51.083656 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.083636 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dznt8\" (UniqueName: \"kubernetes.io/projected/34081b88-e909-4671-a7a4-93ced58d7c3a-kube-api-access-dznt8\") pod \"34081b88-e909-4671-a7a4-93ced58d7c3a\" (UID: \"34081b88-e909-4671-a7a4-93ced58d7c3a\") " Jul 17 10:59:51.083931 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.083821 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:51.083931 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.083833 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdlrx\" (UniqueName: \"kubernetes.io/projected/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-kube-api-access-fdlrx\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:51.083931 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.083843 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c6d33f7-0a85-4a6b-8723-6d1aefdc9689-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:51.084075 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.084054 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-bundle" (OuterVolumeSpecName: "bundle") pod "34081b88-e909-4671-a7a4-93ced58d7c3a" (UID: "34081b88-e909-4671-a7a4-93ced58d7c3a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:59:51.085890 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.085866 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34081b88-e909-4671-a7a4-93ced58d7c3a-kube-api-access-dznt8" (OuterVolumeSpecName: "kube-api-access-dznt8") pod "34081b88-e909-4671-a7a4-93ced58d7c3a" (UID: "34081b88-e909-4671-a7a4-93ced58d7c3a"). InnerVolumeSpecName "kube-api-access-dznt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 10:59:51.091678 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.091653 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-util" (OuterVolumeSpecName: "util") pod "34081b88-e909-4671-a7a4-93ced58d7c3a" (UID: "34081b88-e909-4671-a7a4-93ced58d7c3a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 10:59:51.184729 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.184697 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dznt8\" (UniqueName: \"kubernetes.io/projected/34081b88-e909-4671-a7a4-93ced58d7c3a-kube-api-access-dznt8\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:51.184729 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.184726 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:51.184729 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.184736 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34081b88-e909-4671-a7a4-93ced58d7c3a-util\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 10:59:51.830959 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.830928 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" Jul 17 10:59:51.830959 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.830950 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj" event={"ID":"34081b88-e909-4671-a7a4-93ced58d7c3a","Type":"ContainerDied","Data":"123972068ffc4d5183b75ad85912ee614ec5f8a4d0a655787a21997d8a7c9a8c"} Jul 17 10:59:51.831400 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.830992 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="123972068ffc4d5183b75ad85912ee614ec5f8a4d0a655787a21997d8a7c9a8c" Jul 17 10:59:51.832931 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.832908 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" event={"ID":"5c6d33f7-0a85-4a6b-8723-6d1aefdc9689","Type":"ContainerDied","Data":"398b6d8595910418d973c0c6475e73ed8da19e5035b72038f2dd1117dd006928"} Jul 17 10:59:51.833044 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.832941 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="398b6d8595910418d973c0c6475e73ed8da19e5035b72038f2dd1117dd006928" Jul 17 10:59:51.833044 ip-10-0-143-42 kubenswrapper[2551]: I0717 10:59:51.832952 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz" Jul 17 11:00:13.107382 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107342 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp"] Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107685 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34081b88-e909-4671-a7a4-93ced58d7c3a" containerName="pull" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107697 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="34081b88-e909-4671-a7a4-93ced58d7c3a" containerName="pull" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107708 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" containerName="util" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107713 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" containerName="util" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107719 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0649e1-69e8-46ae-8bf1-ae5442e11997" containerName="extract" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107725 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0649e1-69e8-46ae-8bf1-ae5442e11997" containerName="extract" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107735 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" containerName="extract" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107741 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" containerName="extract" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107748 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" containerName="util" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107753 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" containerName="util" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107763 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" containerName="pull" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107768 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" containerName="pull" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107774 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" containerName="pull" Jul 17 11:00:13.107775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107779 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" containerName="pull" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107786 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0649e1-69e8-46ae-8bf1-ae5442e11997" containerName="pull" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107791 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0649e1-69e8-46ae-8bf1-ae5442e11997" containerName="pull" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107800 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34081b88-e909-4671-a7a4-93ced58d7c3a" containerName="util" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107805 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="34081b88-e909-4671-a7a4-93ced58d7c3a" containerName="util" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107810 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" containerName="extract" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107815 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" containerName="extract" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107820 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34081b88-e909-4671-a7a4-93ced58d7c3a" containerName="extract" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107825 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="34081b88-e909-4671-a7a4-93ced58d7c3a" containerName="extract" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107831 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0649e1-69e8-46ae-8bf1-ae5442e11997" containerName="util" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107836 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0649e1-69e8-46ae-8bf1-ae5442e11997" containerName="util" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107882 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="34081b88-e909-4671-a7a4-93ced58d7c3a" containerName="extract" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107890 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c6d33f7-0a85-4a6b-8723-6d1aefdc9689" containerName="extract" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107899 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9" containerName="extract" Jul 17 11:00:13.108155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.107905 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc0649e1-69e8-46ae-8bf1-ae5442e11997" containerName="extract" Jul 17 11:00:13.110879 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.110862 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:13.114498 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.114471 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 17 11:00:13.114644 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.114471 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bd44q\"" Jul 17 11:00:13.114644 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.114574 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 17 11:00:13.122462 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.122437 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp"] Jul 17 11:00:13.172419 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.172385 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzv2s\" (UniqueName: \"kubernetes.io/projected/d994c6b3-3406-4ccb-9b45-90a5e35e567a-kube-api-access-bzv2s\") pod \"limitador-operator-controller-manager-859d7b78bd-jjrkp\" (UID: \"d994c6b3-3406-4ccb-9b45-90a5e35e567a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:13.272953 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.272904 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzv2s\" (UniqueName: \"kubernetes.io/projected/d994c6b3-3406-4ccb-9b45-90a5e35e567a-kube-api-access-bzv2s\") pod \"limitador-operator-controller-manager-859d7b78bd-jjrkp\" (UID: \"d994c6b3-3406-4ccb-9b45-90a5e35e567a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:13.281810 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.281784 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzv2s\" (UniqueName: \"kubernetes.io/projected/d994c6b3-3406-4ccb-9b45-90a5e35e567a-kube-api-access-bzv2s\") pod \"limitador-operator-controller-manager-859d7b78bd-jjrkp\" (UID: \"d994c6b3-3406-4ccb-9b45-90a5e35e567a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:13.421367 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.421271 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:13.561128 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.561105 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp"] Jul 17 11:00:13.563101 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:00:13.563075 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd994c6b3_3406_4ccb_9b45_90a5e35e567a.slice/crio-8b3744fd68433ad4cbb08fb676164de251ea07818c11bbc7311ba174c17741e5 WatchSource:0}: Error finding container 8b3744fd68433ad4cbb08fb676164de251ea07818c11bbc7311ba174c17741e5: Status 404 returned error can't find the container with id 8b3744fd68433ad4cbb08fb676164de251ea07818c11bbc7311ba174c17741e5 Jul 17 11:00:13.944408 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:13.944367 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" event={"ID":"d994c6b3-3406-4ccb-9b45-90a5e35e567a","Type":"ContainerStarted","Data":"8b3744fd68433ad4cbb08fb676164de251ea07818c11bbc7311ba174c17741e5"} Jul 17 11:00:15.955815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:15.955775 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" event={"ID":"d994c6b3-3406-4ccb-9b45-90a5e35e567a","Type":"ContainerStarted","Data":"978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f"} Jul 17 11:00:15.956275 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:15.955883 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:15.974465 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:15.974391 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" podStartSLOduration=1.106385349 podStartE2EDuration="2.974370543s" totalImagesPullingTime="1.867985194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:00:15.972895361 +0000 UTC m=+479.646411243" watchObservedRunningTime="2026-07-17 11:00:15.974370543 +0000 UTC m=+479.647886196" Jul 17 11:00:24.919170 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:24.919137 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77458bd66c-xmkps"] Jul 17 11:00:24.924252 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:24.924226 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:24.946633 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:24.946596 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77458bd66c-xmkps"] Jul 17 11:00:25.076303 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.076265 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-console-config\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.076303 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.076308 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-oauth-serving-cert\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.076510 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.076333 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-trusted-ca-bundle\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.076510 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.076388 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd262627-495d-4948-894a-3924b0e292b3-console-serving-cert\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.076510 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.076428 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-service-ca\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.076510 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.076462 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd262627-495d-4948-894a-3924b0e292b3-console-oauth-config\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.076510 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.076482 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5rpz\" (UniqueName: \"kubernetes.io/projected/dd262627-495d-4948-894a-3924b0e292b3-kube-api-access-l5rpz\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.177737 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.177649 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-console-config\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.177737 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.177698 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-oauth-serving-cert\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.177737 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.177728 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-trusted-ca-bundle\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.178019 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.177756 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd262627-495d-4948-894a-3924b0e292b3-console-serving-cert\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.178019 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.177778 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-service-ca\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.178019 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.177805 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd262627-495d-4948-894a-3924b0e292b3-console-oauth-config\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.178173 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.178030 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5rpz\" (UniqueName: \"kubernetes.io/projected/dd262627-495d-4948-894a-3924b0e292b3-kube-api-access-l5rpz\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.178656 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.178631 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-oauth-serving-cert\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.178766 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.178672 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-service-ca\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.178766 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.178743 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-console-config\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.178840 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.178786 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd262627-495d-4948-894a-3924b0e292b3-trusted-ca-bundle\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.180342 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.180319 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd262627-495d-4948-894a-3924b0e292b3-console-serving-cert\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.180428 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.180362 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd262627-495d-4948-894a-3924b0e292b3-console-oauth-config\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.197627 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.197596 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5rpz\" (UniqueName: \"kubernetes.io/projected/dd262627-495d-4948-894a-3924b0e292b3-kube-api-access-l5rpz\") pod \"console-77458bd66c-xmkps\" (UID: \"dd262627-495d-4948-894a-3924b0e292b3\") " pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.234842 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.234802 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:25.380524 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:25.380495 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77458bd66c-xmkps"] Jul 17 11:00:25.382126 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:00:25.382089 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd262627_495d_4948_894a_3924b0e292b3.slice/crio-b6f184ec6728691d2e454d0d086a29683c72c57843cd15ad3644bc2036f28407 WatchSource:0}: Error finding container b6f184ec6728691d2e454d0d086a29683c72c57843cd15ad3644bc2036f28407: Status 404 returned error can't find the container with id b6f184ec6728691d2e454d0d086a29683c72c57843cd15ad3644bc2036f28407 Jul 17 11:00:26.008150 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:26.008115 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77458bd66c-xmkps" event={"ID":"dd262627-495d-4948-894a-3924b0e292b3","Type":"ContainerStarted","Data":"318ca136bb0cf58c8817bf41db6474023a7be0d21d423917ab5a2a2c2c0c179c"} Jul 17 11:00:26.008512 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:26.008165 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77458bd66c-xmkps" event={"ID":"dd262627-495d-4948-894a-3924b0e292b3","Type":"ContainerStarted","Data":"b6f184ec6728691d2e454d0d086a29683c72c57843cd15ad3644bc2036f28407"} Jul 17 11:00:26.040697 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:26.040642 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77458bd66c-xmkps" podStartSLOduration=2.040626171 podStartE2EDuration="2.040626171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:00:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:00:26.038380541 +0000 UTC m=+489.711896194" watchObservedRunningTime="2026-07-17 11:00:26.040626171 +0000 UTC m=+489.714141823" Jul 17 11:00:26.963277 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:26.963243 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:26.976227 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:26.976198 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp"] Jul 17 11:00:26.984534 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:26.984502 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp"] Jul 17 11:00:26.994291 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:00:26.994231 2551 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d994c6b3-3406-4ccb-9b45-90a5e35e567a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-17T11:00:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-17T11:00:26Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"200Mi\\\"},\\\"containerID\\\":\\\"cri-o://978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f\\\",\\\"image\\\":\\\"quay.io/kuadrant/limitador-operator:v0.17.1\\\",\\\"imageID\\\":\\\"quay.io/kuadrant/limitador-operator@sha256:3707fe24c6052d5adf240e12d34110706f0ac0877373830b67daab7c9a5c380d\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"manager\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"200Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-17T11:00:15Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000680000],\\\"uid\\\":1000680000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-bzv2s\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-jjrkp\": pods \"limitador-operator-controller-manager-859d7b78bd-jjrkp\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:27.008875 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.008825 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8"] Jul 17 11:00:27.009294 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.009226 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" containerName="manager" Jul 17 11:00:27.009294 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.009239 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" containerName="manager" Jul 17 11:00:27.009409 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.009311 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" containerName="manager" Jul 17 11:00:27.012590 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.012568 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8" Jul 17 11:00:27.015518 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.015486 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" podUID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" containerName="manager" containerID="cri-o://978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f" gracePeriod=2 Jul 17 11:00:27.035460 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.035434 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8"] Jul 17 11:00:27.055128 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:00:27.055086 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jjrkp\" is forbidden: User \"system:node:ip-10-0-143-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-42.ec2.internal' and this object" podUID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:27.058273 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:00:27.058247 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jjrkp\" is forbidden: User \"system:node:ip-10-0-143-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-42.ec2.internal' and this object" podUID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:27.095314 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.095279 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czdhf\" (UniqueName: \"kubernetes.io/projected/0340f437-ce6f-4b84-bc7f-1fc107712c38-kube-api-access-czdhf\") pod \"limitador-operator-controller-manager-859d7b78bd-sb6q8\" (UID: \"0340f437-ce6f-4b84-bc7f-1fc107712c38\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8" Jul 17 11:00:27.196873 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.196839 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czdhf\" (UniqueName: \"kubernetes.io/projected/0340f437-ce6f-4b84-bc7f-1fc107712c38-kube-api-access-czdhf\") pod \"limitador-operator-controller-manager-859d7b78bd-sb6q8\" (UID: \"0340f437-ce6f-4b84-bc7f-1fc107712c38\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8" Jul 17 11:00:27.221599 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.221518 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czdhf\" (UniqueName: \"kubernetes.io/projected/0340f437-ce6f-4b84-bc7f-1fc107712c38-kube-api-access-czdhf\") pod \"limitador-operator-controller-manager-859d7b78bd-sb6q8\" (UID: \"0340f437-ce6f-4b84-bc7f-1fc107712c38\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8" Jul 17 11:00:27.274345 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.274317 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:27.277032 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:00:27.277005 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jjrkp\" is forbidden: User \"system:node:ip-10-0-143-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-42.ec2.internal' and this object" podUID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:27.325368 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.325334 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8" Jul 17 11:00:27.398117 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.398083 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzv2s\" (UniqueName: \"kubernetes.io/projected/d994c6b3-3406-4ccb-9b45-90a5e35e567a-kube-api-access-bzv2s\") pod \"d994c6b3-3406-4ccb-9b45-90a5e35e567a\" (UID: \"d994c6b3-3406-4ccb-9b45-90a5e35e567a\") " Jul 17 11:00:27.400007 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.399966 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d994c6b3-3406-4ccb-9b45-90a5e35e567a-kube-api-access-bzv2s" (OuterVolumeSpecName: "kube-api-access-bzv2s") pod "d994c6b3-3406-4ccb-9b45-90a5e35e567a" (UID: "d994c6b3-3406-4ccb-9b45-90a5e35e567a"). InnerVolumeSpecName "kube-api-access-bzv2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:00:27.463983 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.463929 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8"] Jul 17 11:00:27.466565 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:00:27.466526 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0340f437_ce6f_4b84_bc7f_1fc107712c38.slice/crio-abbdfa122144b71d2dadca226b72c39809f2aa97e31480e6ff48c1666275a7d4 WatchSource:0}: Error finding container abbdfa122144b71d2dadca226b72c39809f2aa97e31480e6ff48c1666275a7d4: Status 404 returned error can't find the container with id abbdfa122144b71d2dadca226b72c39809f2aa97e31480e6ff48c1666275a7d4 Jul 17 11:00:27.499526 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:27.499502 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzv2s\" (UniqueName: \"kubernetes.io/projected/d994c6b3-3406-4ccb-9b45-90a5e35e567a-kube-api-access-bzv2s\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:00:28.021966 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:28.021934 2551 generic.go:356] "Generic (PLEG): container finished" podID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" containerID="978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f" exitCode=0 Jul 17 11:00:28.022417 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:28.021979 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:28.022417 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:28.022035 2551 scope.go:117] "RemoveContainer" containerID="978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f" Jul 17 11:00:28.023779 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:28.023751 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8" event={"ID":"0340f437-ce6f-4b84-bc7f-1fc107712c38","Type":"ContainerStarted","Data":"45022e9f00282169c1cd0df1b4274c15ff157facfcd7666839749c1fd635d87a"} Jul 17 11:00:28.023923 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:28.023805 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8" event={"ID":"0340f437-ce6f-4b84-bc7f-1fc107712c38","Type":"ContainerStarted","Data":"abbdfa122144b71d2dadca226b72c39809f2aa97e31480e6ff48c1666275a7d4"} Jul 17 11:00:28.023923 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:28.023875 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8" Jul 17 11:00:28.025056 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:00:28.025030 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jjrkp\" is forbidden: User \"system:node:ip-10-0-143-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-42.ec2.internal' and this object" podUID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:28.031520 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:28.031505 2551 scope.go:117] "RemoveContainer" containerID="978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f" Jul 17 11:00:28.031795 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:00:28.031777 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f\": container with ID starting with 978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f not found: ID does not exist" containerID="978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f" Jul 17 11:00:28.031846 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:28.031805 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f"} err="failed to get container status \"978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f\": rpc error: code = NotFound desc = could not find container \"978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f\": container with ID starting with 978c602d03a35f81bf1a71b71d7b2df1f70ee3a8f1d4617912a87dc8b2a7568f not found: ID does not exist" Jul 17 11:00:28.057472 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:28.057421 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8" podStartSLOduration=2.057404506 podStartE2EDuration="2.057404506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:00:28.056286223 +0000 UTC m=+491.729801876" watchObservedRunningTime="2026-07-17 11:00:28.057404506 +0000 UTC m=+491.730920157" Jul 17 11:00:28.072609 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:00:28.072569 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jjrkp\" is forbidden: User \"system:node:ip-10-0-143-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-42.ec2.internal' and this object" podUID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:28.077030 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:00:28.077009 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jjrkp\" is forbidden: User \"system:node:ip-10-0-143-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-42.ec2.internal' and this object" podUID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jjrkp" Jul 17 11:00:28.829198 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:28.829156 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d994c6b3-3406-4ccb-9b45-90a5e35e567a" path="/var/lib/kubelet/pods/d994c6b3-3406-4ccb-9b45-90a5e35e567a/volumes" Jul 17 11:00:35.235690 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:35.235642 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:35.235690 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:35.235707 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:35.240726 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:35.240697 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:36.068086 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:36.068049 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77458bd66c-xmkps" Jul 17 11:00:36.133245 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:36.133210 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-798599678b-sjn6n"] Jul 17 11:00:39.031662 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:00:39.031630 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sb6q8" Jul 17 11:01:01.154416 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.154350 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-798599678b-sjn6n" podUID="68a2a055-8a30-41d9-92bd-1577defc3fe7" containerName="console" containerID="cri-o://0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4" gracePeriod=15 Jul 17 11:01:01.402120 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.402092 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798599678b-sjn6n" Jul 17 11:01:01.489513 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.489427 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-serving-cert\") pod \"68a2a055-8a30-41d9-92bd-1577defc3fe7\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " Jul 17 11:01:01.489513 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.489464 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-service-ca\") pod \"68a2a055-8a30-41d9-92bd-1577defc3fe7\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " Jul 17 11:01:01.489513 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.489502 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-oauth-config\") pod \"68a2a055-8a30-41d9-92bd-1577defc3fe7\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " Jul 17 11:01:01.489810 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.489536 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbn2n\" (UniqueName: \"kubernetes.io/projected/68a2a055-8a30-41d9-92bd-1577defc3fe7-kube-api-access-kbn2n\") pod \"68a2a055-8a30-41d9-92bd-1577defc3fe7\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " Jul 17 11:01:01.489810 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.489575 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-oauth-serving-cert\") pod \"68a2a055-8a30-41d9-92bd-1577defc3fe7\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " Jul 17 11:01:01.489810 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.489716 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-trusted-ca-bundle\") pod \"68a2a055-8a30-41d9-92bd-1577defc3fe7\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " Jul 17 11:01:01.489810 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.489777 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-config\") pod \"68a2a055-8a30-41d9-92bd-1577defc3fe7\" (UID: \"68a2a055-8a30-41d9-92bd-1577defc3fe7\") " Jul 17 11:01:01.490082 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.489961 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "68a2a055-8a30-41d9-92bd-1577defc3fe7" (UID: "68a2a055-8a30-41d9-92bd-1577defc3fe7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:01:01.490082 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.489968 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-service-ca" (OuterVolumeSpecName: "service-ca") pod "68a2a055-8a30-41d9-92bd-1577defc3fe7" (UID: "68a2a055-8a30-41d9-92bd-1577defc3fe7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:01:01.490277 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.490190 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "68a2a055-8a30-41d9-92bd-1577defc3fe7" (UID: "68a2a055-8a30-41d9-92bd-1577defc3fe7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:01:01.490372 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.490347 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-config" (OuterVolumeSpecName: "console-config") pod "68a2a055-8a30-41d9-92bd-1577defc3fe7" (UID: "68a2a055-8a30-41d9-92bd-1577defc3fe7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:01:01.491823 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.491795 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "68a2a055-8a30-41d9-92bd-1577defc3fe7" (UID: "68a2a055-8a30-41d9-92bd-1577defc3fe7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 11:01:01.491911 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.491837 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "68a2a055-8a30-41d9-92bd-1577defc3fe7" (UID: "68a2a055-8a30-41d9-92bd-1577defc3fe7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 11:01:01.491911 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.491862 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68a2a055-8a30-41d9-92bd-1577defc3fe7-kube-api-access-kbn2n" (OuterVolumeSpecName: "kube-api-access-kbn2n") pod "68a2a055-8a30-41d9-92bd-1577defc3fe7" (UID: "68a2a055-8a30-41d9-92bd-1577defc3fe7"). InnerVolumeSpecName "kube-api-access-kbn2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:01:01.590839 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.590793 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbn2n\" (UniqueName: \"kubernetes.io/projected/68a2a055-8a30-41d9-92bd-1577defc3fe7-kube-api-access-kbn2n\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:01:01.590839 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.590832 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-oauth-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:01:01.590839 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.590843 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-trusted-ca-bundle\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:01:01.590839 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.590853 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:01:01.591099 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.590862 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-serving-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:01:01.591099 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.590871 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68a2a055-8a30-41d9-92bd-1577defc3fe7-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:01:01.591099 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:01.590879 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68a2a055-8a30-41d9-92bd-1577defc3fe7-console-oauth-config\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:01:02.203023 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:02.202986 2551 generic.go:356] "Generic (PLEG): container finished" podID="68a2a055-8a30-41d9-92bd-1577defc3fe7" containerID="0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4" exitCode=0 Jul 17 11:01:02.203420 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:02.203035 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-798599678b-sjn6n" Jul 17 11:01:02.203420 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:02.203062 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798599678b-sjn6n" event={"ID":"68a2a055-8a30-41d9-92bd-1577defc3fe7","Type":"ContainerDied","Data":"0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4"} Jul 17 11:01:02.203420 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:02.203112 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-798599678b-sjn6n" event={"ID":"68a2a055-8a30-41d9-92bd-1577defc3fe7","Type":"ContainerDied","Data":"c7fd9ed6ccc5deb5fd3dbb9a68d267d9b2a3ae315cb0ccc88535a9be392b8efe"} Jul 17 11:01:02.203420 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:02.203139 2551 scope.go:117] "RemoveContainer" containerID="0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4" Jul 17 11:01:02.212311 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:02.212292 2551 scope.go:117] "RemoveContainer" containerID="0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4" Jul 17 11:01:02.212575 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:01:02.212532 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4\": container with ID starting with 0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4 not found: ID does not exist" containerID="0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4" Jul 17 11:01:02.212673 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:02.212584 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4"} err="failed to get container status \"0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4\": rpc error: code = NotFound desc = could not find container \"0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4\": container with ID starting with 0a286ac46fce8b3be6be7364e8909e76208029d8c5fe790acb54d45c026ea8f4 not found: ID does not exist" Jul 17 11:01:02.225529 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:02.225502 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-798599678b-sjn6n"] Jul 17 11:01:02.229136 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:02.229112 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-798599678b-sjn6n"] Jul 17 11:01:02.823034 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:02.823002 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68a2a055-8a30-41d9-92bd-1577defc3fe7" path="/var/lib/kubelet/pods/68a2a055-8a30-41d9-92bd-1577defc3fe7/volumes" Jul 17 11:01:05.630984 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.630951 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5"] Jul 17 11:01:05.631341 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.631309 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68a2a055-8a30-41d9-92bd-1577defc3fe7" containerName="console" Jul 17 11:01:05.631341 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.631327 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="68a2a055-8a30-41d9-92bd-1577defc3fe7" containerName="console" Jul 17 11:01:05.631411 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.631395 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="68a2a055-8a30-41d9-92bd-1577defc3fe7" containerName="console" Jul 17 11:01:05.635915 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.635895 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.638312 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.638283 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mqw5r\"" Jul 17 11:01:05.649339 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.649309 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5"] Jul 17 11:01:05.726417 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.726374 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5aa62500-a5c0-4773-8aca-059227813d04-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.726638 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.726441 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5aa62500-a5c0-4773-8aca-059227813d04-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.726638 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.726470 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.726638 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.726528 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.726638 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.726581 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.726827 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.726648 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5aa62500-a5c0-4773-8aca-059227813d04-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.726827 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.726705 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nhmp\" (UniqueName: \"kubernetes.io/projected/5aa62500-a5c0-4773-8aca-059227813d04-kube-api-access-5nhmp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.726827 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.726731 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.726827 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.726752 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.827952 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.827911 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5aa62500-a5c0-4773-8aca-059227813d04-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828152 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.827964 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nhmp\" (UniqueName: \"kubernetes.io/projected/5aa62500-a5c0-4773-8aca-059227813d04-kube-api-access-5nhmp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828152 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.827993 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828152 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.828008 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828152 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.828026 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5aa62500-a5c0-4773-8aca-059227813d04-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828152 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.828079 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5aa62500-a5c0-4773-8aca-059227813d04-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828152 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.828117 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828152 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.828149 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828505 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.828187 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828586 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.828503 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828659 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.828584 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828767 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.828741 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828767 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.828756 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5aa62500-a5c0-4773-8aca-059227813d04-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.828913 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.828903 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.830379 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.830356 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5aa62500-a5c0-4773-8aca-059227813d04-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.830590 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.830573 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5aa62500-a5c0-4773-8aca-059227813d04-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.838235 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.838214 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5aa62500-a5c0-4773-8aca-059227813d04-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.838508 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.838488 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nhmp\" (UniqueName: \"kubernetes.io/projected/5aa62500-a5c0-4773-8aca-059227813d04-kube-api-access-5nhmp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-56bm5\" (UID: \"5aa62500-a5c0-4773-8aca-059227813d04\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:05.947460 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:05.947363 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:06.092825 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:06.092794 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5"] Jul 17 11:01:06.095917 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:01:06.095892 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa62500_a5c0_4773_8aca_059227813d04.slice/crio-e74b4771871be95991f4e6c434863426d67e8109910895ca60ec94cc7e35a57b WatchSource:0}: Error finding container e74b4771871be95991f4e6c434863426d67e8109910895ca60ec94cc7e35a57b: Status 404 returned error can't find the container with id e74b4771871be95991f4e6c434863426d67e8109910895ca60ec94cc7e35a57b Jul 17 11:01:06.226811 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:06.226722 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" event={"ID":"5aa62500-a5c0-4773-8aca-059227813d04","Type":"ContainerStarted","Data":"e74b4771871be95991f4e6c434863426d67e8109910895ca60ec94cc7e35a57b"} Jul 17 11:01:08.953977 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:08.953940 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:01:08.954286 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:08.954015 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:01:08.954286 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:08.954043 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:01:09.246327 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:09.246238 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" event={"ID":"5aa62500-a5c0-4773-8aca-059227813d04","Type":"ContainerStarted","Data":"acd49fe48a407cf8f13405d3793f7714fad046e1a92bcdafa6220dbadc1b90ef"} Jul 17 11:01:09.269395 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:09.269337 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" podStartSLOduration=1.413402008 podStartE2EDuration="4.26932077s" totalImagesPullingTime="2.855918762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:01:09.266246316 +0000 UTC m=+532.939761968" watchObservedRunningTime="2026-07-17 11:01:09.26932077 +0000 UTC m=+532.942836417" Jul 17 11:01:09.948004 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:09.947963 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:10.665943 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.665906 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:01:10.669432 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.669411 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" Jul 17 11:01:10.671945 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.671921 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 17 11:01:10.672082 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.671957 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gl6gr\"" Jul 17 11:01:10.682565 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.682514 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:01:10.700482 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.700456 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:01:10.774951 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.774912 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trnch\" (UniqueName: \"kubernetes.io/projected/34164918-cd9a-48fc-b616-2b718997ec7c-kube-api-access-trnch\") pod \"limitador-limitador-54d6d988d5-nkz62\" (UID: \"34164918-cd9a-48fc-b616-2b718997ec7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" Jul 17 11:01:10.775133 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.774982 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34164918-cd9a-48fc-b616-2b718997ec7c-config-file\") pod \"limitador-limitador-54d6d988d5-nkz62\" (UID: \"34164918-cd9a-48fc-b616-2b718997ec7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" Jul 17 11:01:10.875746 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.875712 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34164918-cd9a-48fc-b616-2b718997ec7c-config-file\") pod \"limitador-limitador-54d6d988d5-nkz62\" (UID: \"34164918-cd9a-48fc-b616-2b718997ec7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" Jul 17 11:01:10.875921 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.875790 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trnch\" (UniqueName: \"kubernetes.io/projected/34164918-cd9a-48fc-b616-2b718997ec7c-kube-api-access-trnch\") pod \"limitador-limitador-54d6d988d5-nkz62\" (UID: \"34164918-cd9a-48fc-b616-2b718997ec7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" Jul 17 11:01:10.876371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.876349 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34164918-cd9a-48fc-b616-2b718997ec7c-config-file\") pod \"limitador-limitador-54d6d988d5-nkz62\" (UID: \"34164918-cd9a-48fc-b616-2b718997ec7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" Jul 17 11:01:10.885805 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.885776 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trnch\" (UniqueName: \"kubernetes.io/projected/34164918-cd9a-48fc-b616-2b718997ec7c-kube-api-access-trnch\") pod \"limitador-limitador-54d6d988d5-nkz62\" (UID: \"34164918-cd9a-48fc-b616-2b718997ec7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" Jul 17 11:01:10.952880 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.952807 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:10.979927 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:10.979894 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" Jul 17 11:01:11.117821 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:11.117797 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:01:11.119744 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:01:11.119715 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34164918_cd9a_48fc_b616_2b718997ec7c.slice/crio-be56e92b93db60e6ff2491f7ce5aa28b4d95b1bebe490ef1cfa3395b1d2991c6 WatchSource:0}: Error finding container be56e92b93db60e6ff2491f7ce5aa28b4d95b1bebe490ef1cfa3395b1d2991c6: Status 404 returned error can't find the container with id be56e92b93db60e6ff2491f7ce5aa28b4d95b1bebe490ef1cfa3395b1d2991c6 Jul 17 11:01:11.257834 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:11.257803 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" event={"ID":"34164918-cd9a-48fc-b616-2b718997ec7c","Type":"ContainerStarted","Data":"be56e92b93db60e6ff2491f7ce5aa28b4d95b1bebe490ef1cfa3395b1d2991c6"} Jul 17 11:01:11.258031 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:11.257979 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:11.259006 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:11.258987 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:01:14.276655 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:14.276623 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" event={"ID":"34164918-cd9a-48fc-b616-2b718997ec7c","Type":"ContainerStarted","Data":"d6f89e93d93795bb5f64889fd60da114ad31d4b3fba6549c3fcaea86ca241bbd"} Jul 17 11:01:14.277244 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:14.276749 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" Jul 17 11:01:14.293899 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:14.293850 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" podStartSLOduration=1.6569847800000002 podStartE2EDuration="4.29383434s" totalImagesPullingTime="2.63684956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:01:14.292627623 +0000 UTC m=+537.966143274" watchObservedRunningTime="2026-07-17 11:01:14.29383434 +0000 UTC m=+537.967349992" Jul 17 11:01:25.283735 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:01:25.283707 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkz62" Jul 17 11:02:16.803839 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:16.803803 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:02:16.804360 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:16.803955 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:02:22.665291 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:22.665255 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:02:30.679837 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:30.679806 2551 generic.go:356] "Generic (PLEG): container finished" podID="5aa62500-a5c0-4773-8aca-059227813d04" containerID="acd49fe48a407cf8f13405d3793f7714fad046e1a92bcdafa6220dbadc1b90ef" exitCode=0 Jul 17 11:02:30.680247 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:30.679876 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" event={"ID":"5aa62500-a5c0-4773-8aca-059227813d04","Type":"ContainerDied","Data":"acd49fe48a407cf8f13405d3793f7714fad046e1a92bcdafa6220dbadc1b90ef"} Jul 17 11:02:30.680292 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:30.680278 2551 scope.go:117] "RemoveContainer" containerID="acd49fe48a407cf8f13405d3793f7714fad046e1a92bcdafa6220dbadc1b90ef" Jul 17 11:02:30.681377 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:30.681343 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:02:30.681487 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:30.681413 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:02:30.681487 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:30.681463 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:02:31.687872 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:31.687836 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" event={"ID":"5aa62500-a5c0-4773-8aca-059227813d04","Type":"ContainerStarted","Data":"e54f3f822577ecbbf1066a2ffaa6c9fb99c3ed322ff094d842b55c26ebb7db65"} Jul 17 11:02:31.948215 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:31.948119 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:02:31.948592 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:31.948484 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-56bm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 17 11:02:31.948760 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:31.948738 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" podUID="5aa62500-a5c0-4773-8aca-059227813d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 17 11:02:32.948117 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:32.948077 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-56bm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 17 11:02:32.948505 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:32.948151 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" podUID="5aa62500-a5c0-4773-8aca-059227813d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 17 11:02:33.948287 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:33.948247 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-56bm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 17 11:02:33.948697 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:33.948307 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" podUID="5aa62500-a5c0-4773-8aca-059227813d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 17 11:02:34.948856 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:34.948808 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-56bm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 17 11:02:34.949260 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:34.948873 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" podUID="5aa62500-a5c0-4773-8aca-059227813d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 17 11:02:35.951370 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:35.951337 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:02:35.951772 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:35.951589 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:02:35.952379 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:35.952361 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-56bm5" Jul 17 11:02:53.370900 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:53.370809 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:02:59.369567 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.369516 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s"] Jul 17 11:02:59.373177 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.373155 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.377831 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.377804 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 17 11:02:59.377831 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.377804 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 17 11:02:59.377997 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.377807 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-shjnc\"" Jul 17 11:02:59.377997 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.377848 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 17 11:02:59.387913 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.387878 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s"] Jul 17 11:02:59.547986 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.547949 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.547986 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.547992 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.548235 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.548060 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.548235 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.548080 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.548235 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.548210 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8zg9\" (UniqueName: \"kubernetes.io/projected/7842d7cd-0bde-47ae-a651-64efa592ae4e-kube-api-access-j8zg9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.548377 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.548262 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7842d7cd-0bde-47ae-a651-64efa592ae4e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.649794 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.649702 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.649794 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.649743 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.650025 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.649802 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8zg9\" (UniqueName: \"kubernetes.io/projected/7842d7cd-0bde-47ae-a651-64efa592ae4e-kube-api-access-j8zg9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.650025 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.649835 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7842d7cd-0bde-47ae-a651-64efa592ae4e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.650025 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.649891 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.650025 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.649918 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.650240 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.650155 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.650304 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.650251 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.650304 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.650270 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.652051 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.652029 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7842d7cd-0bde-47ae-a651-64efa592ae4e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.652303 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.652284 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7842d7cd-0bde-47ae-a651-64efa592ae4e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.661769 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.661746 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8zg9\" (UniqueName: \"kubernetes.io/projected/7842d7cd-0bde-47ae-a651-64efa592ae4e-kube-api-access-j8zg9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-btr4s\" (UID: \"7842d7cd-0bde-47ae-a651-64efa592ae4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.683532 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.683486 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:02:59.817884 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.817859 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s"] Jul 17 11:02:59.819709 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:02:59.819685 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7842d7cd_0bde_47ae_a651_64efa592ae4e.slice/crio-068c9931305e27a468a787ae27c1b2f6ccd4c8e58ffb20b4b6a3ceb4720ee846 WatchSource:0}: Error finding container 068c9931305e27a468a787ae27c1b2f6ccd4c8e58ffb20b4b6a3ceb4720ee846: Status 404 returned error can't find the container with id 068c9931305e27a468a787ae27c1b2f6ccd4c8e58ffb20b4b6a3ceb4720ee846 Jul 17 11:02:59.821085 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.821069 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 11:02:59.842168 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:02:59.842142 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" event={"ID":"7842d7cd-0bde-47ae-a651-64efa592ae4e","Type":"ContainerStarted","Data":"068c9931305e27a468a787ae27c1b2f6ccd4c8e58ffb20b4b6a3ceb4720ee846"} Jul 17 11:03:00.255889 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:03:00.255851 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:03:01.858459 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:03:01.858415 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" event={"ID":"7842d7cd-0bde-47ae-a651-64efa592ae4e","Type":"ContainerStarted","Data":"417faaabdb7fc51deb200ce6a2d16cb9a46c9eea2279d3b4bac53bdec8e9358e"} Jul 17 11:03:01.887464 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:03:01.887412 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" podStartSLOduration=1.872242075 podStartE2EDuration="2.887395281s" totalImagesPullingTime="1.015153206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:03:01.88635029 +0000 UTC m=+645.559865942" watchObservedRunningTime="2026-07-17 11:03:01.887395281 +0000 UTC m=+645.560910986" Jul 17 11:03:07.072656 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:03:07.072615 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:03:09.684469 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:03:09.684426 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:03:09.684998 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:03:09.684506 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:03:09.696951 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:03:09.696926 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:03:09.911777 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:03:09.911745 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-btr4s" Jul 17 11:03:22.637131 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:03:22.637079 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:03:33.037107 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:03:33.037071 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:03:40.534891 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:03:40.534852 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:05:00.656930 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:05:00.656894 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:05:10.462700 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:05:10.462662 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:05:21.082899 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:05:21.082860 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:07:04.057155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:07:04.057119 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:07:14.032028 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:07:14.031938 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:07:16.838494 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:07:16.838463 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:07:16.839770 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:07:16.839747 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:07:26.736469 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:07:26.736434 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:07:43.056796 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:07:43.052588 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:07:50.956166 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:07:50.956126 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:08:01.440874 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:08:01.440838 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:08:09.233074 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:08:09.233037 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:08:20.649081 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:08:20.649037 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:08:29.742408 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:08:29.742365 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:08:41.942811 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:08:41.942728 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:08:51.043201 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:08:51.043162 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:09:13.739493 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:09:13.739455 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:09:25.643330 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:09:25.643291 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:09:49.040675 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:09:49.040632 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:10:00.250459 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:10:00.250419 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:10:15.062921 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:10:15.062835 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:10:26.333897 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:10:26.333859 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:10:35.643017 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:10:35.642982 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:10:44.634420 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:10:44.634381 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:10:52.939757 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:10:52.939718 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:12:16.872839 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:12:16.872808 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:12:16.874914 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:12:16.874892 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:12:43.044030 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:12:43.043988 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:12:52.233881 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:12:52.233843 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:13:04.450897 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:13:04.450855 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:13:16.243746 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:13:16.243661 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:13:27.137281 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:13:27.137245 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:13:37.544525 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:13:37.544482 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:13:46.552629 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:13:46.552581 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:13:57.340682 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:13:57.340648 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:14:07.438420 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:14:07.438389 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:14:17.843105 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:14:17.843070 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:14:28.945016 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:14:28.944980 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:14:52.537932 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:14:52.537894 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:14:59.645388 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:14:59.645350 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:15:08.636286 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:15:08.636245 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:15:20.534072 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:15:20.534036 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:15:35.941382 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:15:35.941343 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:15:44.539037 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:15:44.538997 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:15:53.136795 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:15:53.136758 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:16:01.652436 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:16:01.652396 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:16:11.644627 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:16:11.644523 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:16:30.447696 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:16:30.447658 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:16:38.742649 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:16:38.742608 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:16:50.035452 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:16:50.035416 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:16:58.855726 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:16:58.855690 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:17:10.460903 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:17:10.460862 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:17:16.910810 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:17:16.910758 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:17:16.913473 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:17:16.913452 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:17:22.856593 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:17:22.856541 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:17:33.452782 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:17:33.452742 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:17:43.659035 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:17:43.658994 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:17:55.343935 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:17:55.343902 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:18:05.534208 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:18:05.534174 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:18:25.738434 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:18:25.738398 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:21:23.022214 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.022175 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8"] Jul 17 11:21:23.025534 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.025517 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" Jul 17 11:21:23.028150 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.028128 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 17 11:21:23.028263 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.028224 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87tmr\"" Jul 17 11:21:23.034250 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.034225 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8"] Jul 17 11:21:23.070609 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.070570 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7703e7f4-f3d0-48f8-9511-133a2e8069f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0839e276-d8gd8\" (UID: \"7703e7f4-f3d0-48f8-9511-133a2e8069f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" Jul 17 11:21:23.171456 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.171413 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7703e7f4-f3d0-48f8-9511-133a2e8069f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0839e276-d8gd8\" (UID: \"7703e7f4-f3d0-48f8-9511-133a2e8069f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" Jul 17 11:21:23.172066 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.172043 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7703e7f4-f3d0-48f8-9511-133a2e8069f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0839e276-d8gd8\" (UID: \"7703e7f4-f3d0-48f8-9511-133a2e8069f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" Jul 17 11:21:23.336934 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.336850 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" Jul 17 11:21:23.472718 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.472689 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8"] Jul 17 11:21:23.473740 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:21:23.473714 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7703e7f4_f3d0_48f8_9511_133a2e8069f7.slice/crio-d75b2f7cc3e579882d78652e6ba5612df420fde5335f22fcd98719c566e5df2f WatchSource:0}: Error finding container d75b2f7cc3e579882d78652e6ba5612df420fde5335f22fcd98719c566e5df2f: Status 404 returned error can't find the container with id d75b2f7cc3e579882d78652e6ba5612df420fde5335f22fcd98719c566e5df2f Jul 17 11:21:23.476000 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.475984 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 11:21:23.691312 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:23.691216 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" event={"ID":"7703e7f4-f3d0-48f8-9511-133a2e8069f7","Type":"ContainerStarted","Data":"d75b2f7cc3e579882d78652e6ba5612df420fde5335f22fcd98719c566e5df2f"} Jul 17 11:21:26.560915 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:26.560873 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:21:26.709429 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:26.709391 2551 generic.go:356] "Generic (PLEG): container finished" podID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerID="5c836468e9d95f4a804bdefff955713ee197cf9e366aac2632a034cf339223a0" exitCode=7 Jul 17 11:21:26.709651 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:26.709481 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" event={"ID":"7703e7f4-f3d0-48f8-9511-133a2e8069f7","Type":"ContainerDied","Data":"5c836468e9d95f4a804bdefff955713ee197cf9e366aac2632a034cf339223a0"} Jul 17 11:21:26.709792 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:26.709692 2551 scope.go:117] "RemoveContainer" containerID="5c836468e9d95f4a804bdefff955713ee197cf9e366aac2632a034cf339223a0" Jul 17 11:21:27.717026 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:27.716932 2551 generic.go:356] "Generic (PLEG): container finished" podID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerID="ccb4288a376b7ff968ec207c5350d9de754872e0bc00ed5e44994aec44567ef2" exitCode=7 Jul 17 11:21:27.717026 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:27.716983 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" event={"ID":"7703e7f4-f3d0-48f8-9511-133a2e8069f7","Type":"ContainerDied","Data":"ccb4288a376b7ff968ec207c5350d9de754872e0bc00ed5e44994aec44567ef2"} Jul 17 11:21:27.717531 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:27.717029 2551 scope.go:117] "RemoveContainer" containerID="5c836468e9d95f4a804bdefff955713ee197cf9e366aac2632a034cf339223a0" Jul 17 11:21:27.717531 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:27.717235 2551 scope.go:117] "RemoveContainer" containerID="ccb4288a376b7ff968ec207c5350d9de754872e0bc00ed5e44994aec44567ef2" Jul 17 11:21:27.717531 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:21:27.717486 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0839e276-d8gd8_odh-ai-gateway-infra(7703e7f4-f3d0-48f8-9511-133a2e8069f7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" Jul 17 11:21:28.724780 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:28.724756 2551 scope.go:117] "RemoveContainer" containerID="ccb4288a376b7ff968ec207c5350d9de754872e0bc00ed5e44994aec44567ef2" Jul 17 11:21:28.725238 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:21:28.724945 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0839e276-d8gd8_odh-ai-gateway-infra(7703e7f4-f3d0-48f8-9511-133a2e8069f7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" Jul 17 11:21:41.817613 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:41.817578 2551 scope.go:117] "RemoveContainer" containerID="ccb4288a376b7ff968ec207c5350d9de754872e0bc00ed5e44994aec44567ef2" Jul 17 11:21:42.441688 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:42.441593 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m"] Jul 17 11:21:42.445100 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:42.445077 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" Jul 17 11:21:42.456583 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:42.456539 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m"] Jul 17 11:21:42.551223 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:42.551179 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m\" (UID: \"ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" Jul 17 11:21:42.652499 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:42.652461 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m\" (UID: \"ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" Jul 17 11:21:42.653126 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:42.653104 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m\" (UID: \"ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" Jul 17 11:21:42.756595 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:42.756565 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" Jul 17 11:21:42.799397 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:42.799365 2551 generic.go:356] "Generic (PLEG): container finished" podID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerID="c5018e4dbccd31be8328ff7f83b7a9b54b29a9edd680e0b7143fbb769df6941b" exitCode=6 Jul 17 11:21:42.799612 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:42.799440 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" event={"ID":"7703e7f4-f3d0-48f8-9511-133a2e8069f7","Type":"ContainerDied","Data":"c5018e4dbccd31be8328ff7f83b7a9b54b29a9edd680e0b7143fbb769df6941b"} Jul 17 11:21:42.799612 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:42.799512 2551 scope.go:117] "RemoveContainer" containerID="ccb4288a376b7ff968ec207c5350d9de754872e0bc00ed5e44994aec44567ef2" Jul 17 11:21:42.799889 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:42.799873 2551 scope.go:117] "RemoveContainer" containerID="c5018e4dbccd31be8328ff7f83b7a9b54b29a9edd680e0b7143fbb769df6941b" Jul 17 11:21:42.800127 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:21:42.800108 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0839e276-d8gd8_odh-ai-gateway-infra(7703e7f4-f3d0-48f8-9511-133a2e8069f7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" Jul 17 11:21:43.112390 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:43.112362 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m"] Jul 17 11:21:43.114009 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:21:43.113984 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae9c5c38_e4c5_4d97_a5d2_e90fb13aec43.slice/crio-221e75d6efb91be8a4b5b0d5dd35a194bfa36d19a22d3b92f5860200254f4fca WatchSource:0}: Error finding container 221e75d6efb91be8a4b5b0d5dd35a194bfa36d19a22d3b92f5860200254f4fca: Status 404 returned error can't find the container with id 221e75d6efb91be8a4b5b0d5dd35a194bfa36d19a22d3b92f5860200254f4fca Jul 17 11:21:43.809837 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:43.809807 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" event={"ID":"ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43","Type":"ContainerStarted","Data":"28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd"} Jul 17 11:21:43.809960 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:43.809857 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" event={"ID":"ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43","Type":"ContainerStarted","Data":"221e75d6efb91be8a4b5b0d5dd35a194bfa36d19a22d3b92f5860200254f4fca"} Jul 17 11:21:43.810019 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:43.810002 2551 scope.go:117] "RemoveContainer" containerID="28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd" Jul 17 11:21:43.831921 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:43.831896 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8"] Jul 17 11:21:43.962073 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:43.962049 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" Jul 17 11:21:44.067119 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.067029 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7703e7f4-f3d0-48f8-9511-133a2e8069f7-maas-api-service-ca\") pod \"7703e7f4-f3d0-48f8-9511-133a2e8069f7\" (UID: \"7703e7f4-f3d0-48f8-9511-133a2e8069f7\") " Jul 17 11:21:44.067380 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.067356 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7703e7f4-f3d0-48f8-9511-133a2e8069f7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7703e7f4-f3d0-48f8-9511-133a2e8069f7" (UID: "7703e7f4-f3d0-48f8-9511-133a2e8069f7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:21:44.167815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.167778 2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7703e7f4-f3d0-48f8-9511-133a2e8069f7-maas-api-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:21:44.816477 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.816435 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" Jul 17 11:21:44.816699 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.816473 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8" event={"ID":"7703e7f4-f3d0-48f8-9511-133a2e8069f7","Type":"ContainerDied","Data":"d75b2f7cc3e579882d78652e6ba5612df420fde5335f22fcd98719c566e5df2f"} Jul 17 11:21:44.816699 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.816602 2551 scope.go:117] "RemoveContainer" containerID="c5018e4dbccd31be8328ff7f83b7a9b54b29a9edd680e0b7143fbb769df6941b" Jul 17 11:21:44.818679 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.818658 2551 generic.go:356] "Generic (PLEG): container finished" podID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerID="28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd" exitCode=7 Jul 17 11:21:44.818679 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.818680 2551 generic.go:356] "Generic (PLEG): container finished" podID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerID="6200f6b69a01305b5a37951fafab6c8bea3bc284364f4bd4ceb46a9a3510741e" exitCode=7 Jul 17 11:21:44.819022 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.819006 2551 scope.go:117] "RemoveContainer" containerID="6200f6b69a01305b5a37951fafab6c8bea3bc284364f4bd4ceb46a9a3510741e" Jul 17 11:21:44.819289 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:21:44.819265 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m_odh-ai-gateway-infra(ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" Jul 17 11:21:44.826366 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.826333 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" event={"ID":"ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43","Type":"ContainerDied","Data":"28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd"} Jul 17 11:21:44.826498 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.826397 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" event={"ID":"ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43","Type":"ContainerDied","Data":"6200f6b69a01305b5a37951fafab6c8bea3bc284364f4bd4ceb46a9a3510741e"} Jul 17 11:21:44.827263 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.827242 2551 scope.go:117] "RemoveContainer" containerID="28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd" Jul 17 11:21:44.841245 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.840933 2551 scope.go:117] "RemoveContainer" containerID="28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd" Jul 17 11:21:44.841500 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:21:44.841471 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd\": container with ID starting with 28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd not found: ID does not exist" containerID="28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd" Jul 17 11:21:44.842255 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.841510 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd"} err="failed to get container status \"28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd\": rpc error: code = NotFound desc = could not find container \"28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd\": container with ID starting with 28fe7cd1b61b4397007c496e9d9085906e995802b8f38e5ab6b4eabe3813bfbd not found: ID does not exist" Jul 17 11:21:44.858398 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.857093 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8"] Jul 17 11:21:44.861414 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:44.861390 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0839e276-d8gd8"] Jul 17 11:21:45.827142 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:45.827110 2551 scope.go:117] "RemoveContainer" containerID="6200f6b69a01305b5a37951fafab6c8bea3bc284364f4bd4ceb46a9a3510741e" Jul 17 11:21:45.827733 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:21:45.827342 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m_odh-ai-gateway-infra(ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" Jul 17 11:21:46.335606 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:46.335566 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:21:46.823713 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:21:46.823682 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" path="/var/lib/kubelet/pods/7703e7f4-f3d0-48f8-9511-133a2e8069f7/volumes" Jul 17 11:22:00.817928 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:00.817894 2551 scope.go:117] "RemoveContainer" containerID="6200f6b69a01305b5a37951fafab6c8bea3bc284364f4bd4ceb46a9a3510741e" Jul 17 11:22:01.921361 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:01.921313 2551 generic.go:356] "Generic (PLEG): container finished" podID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerID="51a78ac28b8a9bcecf93eaddeca093da0c50088e3ee7f5f06141f2d06e978f95" exitCode=7 Jul 17 11:22:01.921817 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:01.921385 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" event={"ID":"ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43","Type":"ContainerDied","Data":"51a78ac28b8a9bcecf93eaddeca093da0c50088e3ee7f5f06141f2d06e978f95"} Jul 17 11:22:01.921817 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:01.921467 2551 scope.go:117] "RemoveContainer" containerID="6200f6b69a01305b5a37951fafab6c8bea3bc284364f4bd4ceb46a9a3510741e" Jul 17 11:22:01.921817 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:01.921732 2551 scope.go:117] "RemoveContainer" containerID="51a78ac28b8a9bcecf93eaddeca093da0c50088e3ee7f5f06141f2d06e978f95" Jul 17 11:22:01.921960 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:22:01.921942 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m_odh-ai-gateway-infra(ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" Jul 17 11:22:02.948421 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:02.948389 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m"] Jul 17 11:22:03.084838 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:03.084813 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" Jul 17 11:22:03.132673 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:03.132644 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43-maas-api-service-ca\") pod \"ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43\" (UID: \"ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43\") " Jul 17 11:22:03.133040 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:03.133016 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" (UID: "ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:22:03.233404 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:03.233310 2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43-maas-api-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:22:03.933880 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:03.933849 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" Jul 17 11:22:03.934086 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:03.933895 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m" event={"ID":"ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43","Type":"ContainerDied","Data":"221e75d6efb91be8a4b5b0d5dd35a194bfa36d19a22d3b92f5860200254f4fca"} Jul 17 11:22:03.934086 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:03.933969 2551 scope.go:117] "RemoveContainer" containerID="51a78ac28b8a9bcecf93eaddeca093da0c50088e3ee7f5f06141f2d06e978f95" Jul 17 11:22:03.956221 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:03.956190 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m"] Jul 17 11:22:03.962603 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:03.962579 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4d5573f1-h7h6m"] Jul 17 11:22:04.823500 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:04.823465 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" path="/var/lib/kubelet/pods/ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43/volumes" Jul 17 11:22:16.951328 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:16.951298 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:22:16.956092 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:16.956066 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:22:46.245333 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:22:46.245300 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:23:05.050033 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:05.049999 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:23:52.213906 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.213825 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28"] Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214211 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214224 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214239 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214245 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214259 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214265 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214273 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214278 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214285 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214290 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214348 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214355 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214361 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214368 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerName="revoke-keys" Jul 17 11:23:52.214371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.214374 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="7703e7f4-f3d0-48f8-9511-133a2e8069f7" containerName="revoke-keys" Jul 17 11:23:52.217430 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.217413 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" Jul 17 11:23:52.221481 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.221458 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 17 11:23:52.221584 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.221489 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87tmr\"" Jul 17 11:23:52.226773 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.226749 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28"] Jul 17 11:23:52.313620 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.313589 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c070665-6318-4d05-8b83-1fdfda6e15e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28\" (UID: \"5c070665-6318-4d05-8b83-1fdfda6e15e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" Jul 17 11:23:52.414939 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.414901 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c070665-6318-4d05-8b83-1fdfda6e15e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28\" (UID: \"5c070665-6318-4d05-8b83-1fdfda6e15e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" Jul 17 11:23:52.415572 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.415523 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c070665-6318-4d05-8b83-1fdfda6e15e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28\" (UID: \"5c070665-6318-4d05-8b83-1fdfda6e15e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" Jul 17 11:23:52.529953 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.529915 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" Jul 17 11:23:52.664442 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:52.664413 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28"] Jul 17 11:23:52.666448 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:23:52.666421 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c070665_6318_4d05_8b83_1fdfda6e15e6.slice/crio-9c1f629302646126f6a04d50ea30a6a46387f2aee7878e03ed312e36b1c0ed62 WatchSource:0}: Error finding container 9c1f629302646126f6a04d50ea30a6a46387f2aee7878e03ed312e36b1c0ed62: Status 404 returned error can't find the container with id 9c1f629302646126f6a04d50ea30a6a46387f2aee7878e03ed312e36b1c0ed62 Jul 17 11:23:53.520360 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:53.520316 2551 generic.go:356] "Generic (PLEG): container finished" podID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerID="ec4bac4dea7e9a32612dd4c6e7f8f2f385f4f67bd653f0882717a6f10d176001" exitCode=7 Jul 17 11:23:53.520778 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:53.520405 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" event={"ID":"5c070665-6318-4d05-8b83-1fdfda6e15e6","Type":"ContainerDied","Data":"ec4bac4dea7e9a32612dd4c6e7f8f2f385f4f67bd653f0882717a6f10d176001"} Jul 17 11:23:53.520778 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:53.520463 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" event={"ID":"5c070665-6318-4d05-8b83-1fdfda6e15e6","Type":"ContainerStarted","Data":"9c1f629302646126f6a04d50ea30a6a46387f2aee7878e03ed312e36b1c0ed62"} Jul 17 11:23:53.520778 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:53.520597 2551 scope.go:117] "RemoveContainer" containerID="ec4bac4dea7e9a32612dd4c6e7f8f2f385f4f67bd653f0882717a6f10d176001" Jul 17 11:23:54.529277 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:54.527891 2551 generic.go:356] "Generic (PLEG): container finished" podID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerID="dae92bc2362291097fa58f49786cef894f13f9dc8d238fe51544a88164936f08" exitCode=7 Jul 17 11:23:54.529277 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:54.527970 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" event={"ID":"5c070665-6318-4d05-8b83-1fdfda6e15e6","Type":"ContainerDied","Data":"dae92bc2362291097fa58f49786cef894f13f9dc8d238fe51544a88164936f08"} Jul 17 11:23:54.529277 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:54.528027 2551 scope.go:117] "RemoveContainer" containerID="ec4bac4dea7e9a32612dd4c6e7f8f2f385f4f67bd653f0882717a6f10d176001" Jul 17 11:23:54.529277 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:54.528439 2551 scope.go:117] "RemoveContainer" containerID="dae92bc2362291097fa58f49786cef894f13f9dc8d238fe51544a88164936f08" Jul 17 11:23:54.529277 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:23:54.528706 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28_odh-ai-gateway-infra(5c070665-6318-4d05-8b83-1fdfda6e15e6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" Jul 17 11:23:55.535246 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:23:55.535219 2551 scope.go:117] "RemoveContainer" containerID="dae92bc2362291097fa58f49786cef894f13f9dc8d238fe51544a88164936f08" Jul 17 11:23:55.535647 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:23:55.535399 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28_odh-ai-gateway-infra(5c070665-6318-4d05-8b83-1fdfda6e15e6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" Jul 17 11:24:07.816889 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:07.816853 2551 scope.go:117] "RemoveContainer" containerID="dae92bc2362291097fa58f49786cef894f13f9dc8d238fe51544a88164936f08" Jul 17 11:24:08.604365 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:08.604326 2551 generic.go:356] "Generic (PLEG): container finished" podID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerID="e66fb278fa54f239524a94df1a4397d1d0eb44534b92abacb421b457f4c35fdd" exitCode=7 Jul 17 11:24:08.604582 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:08.604405 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" event={"ID":"5c070665-6318-4d05-8b83-1fdfda6e15e6","Type":"ContainerDied","Data":"e66fb278fa54f239524a94df1a4397d1d0eb44534b92abacb421b457f4c35fdd"} Jul 17 11:24:08.604582 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:08.604507 2551 scope.go:117] "RemoveContainer" containerID="dae92bc2362291097fa58f49786cef894f13f9dc8d238fe51544a88164936f08" Jul 17 11:24:08.604867 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:08.604852 2551 scope.go:117] "RemoveContainer" containerID="e66fb278fa54f239524a94df1a4397d1d0eb44534b92abacb421b457f4c35fdd" Jul 17 11:24:08.605078 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:24:08.605056 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28_odh-ai-gateway-infra(5c070665-6318-4d05-8b83-1fdfda6e15e6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" Jul 17 11:24:09.639469 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:09.639432 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28"] Jul 17 11:24:09.778523 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:09.778496 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" Jul 17 11:24:09.877348 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:09.877306 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c070665-6318-4d05-8b83-1fdfda6e15e6-maas-api-service-ca\") pod \"5c070665-6318-4d05-8b83-1fdfda6e15e6\" (UID: \"5c070665-6318-4d05-8b83-1fdfda6e15e6\") " Jul 17 11:24:09.877763 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:09.877735 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c070665-6318-4d05-8b83-1fdfda6e15e6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5c070665-6318-4d05-8b83-1fdfda6e15e6" (UID: "5c070665-6318-4d05-8b83-1fdfda6e15e6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:24:09.979138 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:09.979036 2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c070665-6318-4d05-8b83-1fdfda6e15e6-maas-api-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:24:10.617771 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:10.617732 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" Jul 17 11:24:10.617955 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:10.617774 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28" event={"ID":"5c070665-6318-4d05-8b83-1fdfda6e15e6","Type":"ContainerDied","Data":"9c1f629302646126f6a04d50ea30a6a46387f2aee7878e03ed312e36b1c0ed62"} Jul 17 11:24:10.617955 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:10.617842 2551 scope.go:117] "RemoveContainer" containerID="e66fb278fa54f239524a94df1a4397d1d0eb44534b92abacb421b457f4c35fdd" Jul 17 11:24:10.646916 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:10.646879 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28"] Jul 17 11:24:10.649470 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:10.649441 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eac6ba90-5fh28"] Jul 17 11:24:10.829883 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:24:10.829848 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" path="/var/lib/kubelet/pods/5c070665-6318-4d05-8b83-1fdfda6e15e6/volumes" Jul 17 11:27:16.989266 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:27:16.989213 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:27:16.995242 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:27:16.995216 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:29:06.947100 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:06.947056 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:29:16.644382 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.644343 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj"] Jul 17 11:29:16.644954 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.644933 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerName="revoke-keys" Jul 17 11:29:16.645041 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.644956 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerName="revoke-keys" Jul 17 11:29:16.645041 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.644973 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerName="revoke-keys" Jul 17 11:29:16.645041 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.644980 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerName="revoke-keys" Jul 17 11:29:16.645041 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.644990 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerName="revoke-keys" Jul 17 11:29:16.645041 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.644998 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerName="revoke-keys" Jul 17 11:29:16.645041 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.645029 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerName="revoke-keys" Jul 17 11:29:16.645041 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.645038 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerName="revoke-keys" Jul 17 11:29:16.645465 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.645127 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerName="revoke-keys" Jul 17 11:29:16.645465 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.645141 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerName="revoke-keys" Jul 17 11:29:16.645465 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.645149 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c070665-6318-4d05-8b83-1fdfda6e15e6" containerName="revoke-keys" Jul 17 11:29:16.645465 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.645158 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae9c5c38-e4c5-4d97-a5d2-e90fb13aec43" containerName="revoke-keys" Jul 17 11:29:16.648997 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.648979 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.651630 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.651609 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-71832244-openshift-default-dockercfg-5wfz7\"" Jul 17 11:29:16.665838 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.665812 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj"] Jul 17 11:29:16.749872 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.749835 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-podinfo\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.749872 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.749878 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxh88\" (UniqueName: \"kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-kube-api-access-qxh88\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.750111 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.749994 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-token\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.750111 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.750039 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-data\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.750111 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.750086 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-envoy\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.750284 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.750132 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-credential-socket\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.750284 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.750159 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istiod-ca-cert\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.750284 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.750192 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-certs\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.750284 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.750227 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-socket\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.851571 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.851495 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-token\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.851571 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.851576 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-data\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.851828 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.851607 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-envoy\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.851828 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.851631 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-credential-socket\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.851828 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.851660 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istiod-ca-cert\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.851828 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.851690 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-certs\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.851828 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.851716 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-socket\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.852099 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.851845 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-podinfo\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.852099 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.851874 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxh88\" (UniqueName: \"kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-kube-api-access-qxh88\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.852099 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.852003 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-credential-socket\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.852099 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.852056 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-data\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.852277 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.852169 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-socket\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.852277 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.852244 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-certs\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.852436 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.852396 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istiod-ca-cert\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.853853 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.853829 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-envoy\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.854110 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.854084 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-podinfo\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.872210 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.872181 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxh88\" (UniqueName: \"kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-kube-api-access-qxh88\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.874266 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.874235 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-token\") pod \"e2e-derive-71832244-openshift-default-6887bd4c9f-szswj\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:16.962342 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.962270 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-71832244-openshift-default-dockercfg-5wfz7\"" Jul 17 11:29:16.969695 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:16.969673 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:17.128729 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:17.128703 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj"] Jul 17 11:29:17.130815 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:29:17.130790 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b1a63c_6eb2_4962_8925_b185fdbcc7ea.slice/crio-37a07e79e39303a2cb83893f2f424cd08a8f698c3821dc842aa17e5550b817c0 WatchSource:0}: Error finding container 37a07e79e39303a2cb83893f2f424cd08a8f698c3821dc842aa17e5550b817c0: Status 404 returned error can't find the container with id 37a07e79e39303a2cb83893f2f424cd08a8f698c3821dc842aa17e5550b817c0 Jul 17 11:29:17.132833 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:17.132800 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:29:17.132936 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:17.132874 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:29:17.132936 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:17.132916 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:29:17.240215 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:17.240129 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" event={"ID":"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea","Type":"ContainerStarted","Data":"2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837"} Jul 17 11:29:17.240215 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:17.240181 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" event={"ID":"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea","Type":"ContainerStarted","Data":"37a07e79e39303a2cb83893f2f424cd08a8f698c3821dc842aa17e5550b817c0"} Jul 17 11:29:17.265703 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:17.265653 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" podStartSLOduration=1.265633882 podStartE2EDuration="1.265633882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:29:17.264533176 +0000 UTC m=+2220.938048820" watchObservedRunningTime="2026-07-17 11:29:17.265633882 +0000 UTC m=+2220.939149534" Jul 17 11:29:17.970706 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:17.970669 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:18.997982 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:18.997941 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:19.250925 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:19.250902 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:19.252015 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:19.251989 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:25.744846 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:25.744807 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:29:34.252513 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:34.252460 2551 patch_prober.go:28] interesting pod/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 17 11:29:34.252977 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:34.252537 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 17 11:29:34.344230 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:34.344199 2551 generic.go:356] "Generic (PLEG): container finished" podID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerID="2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837" exitCode=0 Jul 17 11:29:34.344412 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:34.344246 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" event={"ID":"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea","Type":"ContainerDied","Data":"2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837"} Jul 17 11:29:34.344653 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:34.344637 2551 scope.go:117] "RemoveContainer" containerID="2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837" Jul 17 11:29:34.345800 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:34.345769 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:29:34.345898 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:34.345843 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:29:34.345898 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:34.345880 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:29:35.351475 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:35.351436 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" event={"ID":"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea","Type":"ContainerStarted","Data":"d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d"} Jul 17 11:29:35.970124 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:35.970084 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:35.970352 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:35.970325 2551 patch_prober.go:28] interesting pod/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 17 11:29:35.970429 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:35.970386 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 17 11:29:36.970817 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:36.970783 2551 patch_prober.go:28] interesting pod/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 17 11:29:36.971199 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:36.970837 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 17 11:29:37.970082 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:37.970042 2551 patch_prober.go:28] interesting pod/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 17 11:29:37.970340 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:37.970117 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 17 11:29:38.970537 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:38.970443 2551 patch_prober.go:28] interesting pod/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 17 11:29:38.970537 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:38.970499 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 17 11:29:39.974696 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:39.974656 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:39.975156 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:39.975138 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:29:39.975936 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:29:39.975920 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:32:17.025844 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:17.025812 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:32:17.033050 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:17.033026 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:32:23.317763 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:23.317728 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj"] Jul 17 11:32:23.318172 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:23.317947 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" containerID="cri-o://d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d" gracePeriod=30 Jul 17 11:32:24.977803 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:24.977761 2551 patch_prober.go:28] interesting pod/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 11:32:24.978248 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:24.977826 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 11:32:26.336608 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:26.336575 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:32:28.580627 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.580563 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:32:28.688328 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688296 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-certs\") pod \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " Jul 17 11:32:28.688502 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688338 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istiod-ca-cert\") pod \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " Jul 17 11:32:28.688502 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688363 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-socket\") pod \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " Jul 17 11:32:28.688502 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688401 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-podinfo\") pod \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " Jul 17 11:32:28.688502 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688453 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-credential-socket\") pod \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " Jul 17 11:32:28.688502 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688484 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-envoy\") pod \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " Jul 17 11:32:28.688788 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688624 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxh88\" (UniqueName: \"kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-kube-api-access-qxh88\") pod \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " Jul 17 11:32:28.688788 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688650 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" (UID: "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:32:28.688788 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688711 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-token\") pod \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " Jul 17 11:32:28.688788 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688784 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-data\") pod \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\" (UID: \"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea\") " Jul 17 11:32:28.689017 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688850 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" (UID: "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:32:28.689017 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688883 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" (UID: "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:32:28.689130 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.689112 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-certs\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:32:28.689184 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.689133 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istiod-ca-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:32:28.689184 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.689146 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-credential-socket\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:32:28.689184 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.688811 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" (UID: "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:32:28.689909 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.689885 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-data" (OuterVolumeSpecName: "istio-data") pod "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" (UID: "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:32:28.690922 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.690897 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" (UID: "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:32:28.691017 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.690897 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-kube-api-access-qxh88" (OuterVolumeSpecName: "kube-api-access-qxh88") pod "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" (UID: "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea"). InnerVolumeSpecName "kube-api-access-qxh88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:32:28.691081 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.691058 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" (UID: "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 11:32:28.691138 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.691072 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-token" (OuterVolumeSpecName: "istio-token") pod "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" (UID: "b0b1a63c-6eb2-4962-8925-b185fdbcc7ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:32:28.790368 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.790332 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-podinfo\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:32:28.790368 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.790367 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-envoy\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:32:28.790616 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.790379 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxh88\" (UniqueName: \"kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-kube-api-access-qxh88\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:32:28.790616 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.790392 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-token\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:32:28.790616 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.790404 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-istio-data\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:32:28.790616 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:28.790414 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea-workload-socket\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:32:29.284393 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.284359 2551 generic.go:356] "Generic (PLEG): container finished" podID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerID="d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d" exitCode=0 Jul 17 11:32:29.284621 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.284416 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" Jul 17 11:32:29.284621 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.284426 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" event={"ID":"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea","Type":"ContainerDied","Data":"d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d"} Jul 17 11:32:29.284621 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.284470 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj" event={"ID":"b0b1a63c-6eb2-4962-8925-b185fdbcc7ea","Type":"ContainerDied","Data":"37a07e79e39303a2cb83893f2f424cd08a8f698c3821dc842aa17e5550b817c0"} Jul 17 11:32:29.284621 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.284492 2551 scope.go:117] "RemoveContainer" containerID="d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d" Jul 17 11:32:29.294684 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.294656 2551 scope.go:117] "RemoveContainer" containerID="2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837" Jul 17 11:32:29.307576 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.307526 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj"] Jul 17 11:32:29.307936 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.307914 2551 scope.go:117] "RemoveContainer" containerID="d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d" Jul 17 11:32:29.308476 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:32:29.308443 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d\": container with ID starting with d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d not found: ID does not exist" containerID="d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d" Jul 17 11:32:29.308574 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.308488 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d"} err="failed to get container status \"d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d\": rpc error: code = NotFound desc = could not find container \"d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d\": container with ID starting with d3464e5c9fb2fa484151817021b87b5a4e56dc0dd9e4a8661a68b67f2f5bff9d not found: ID does not exist" Jul 17 11:32:29.308574 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.308516 2551 scope.go:117] "RemoveContainer" containerID="2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837" Jul 17 11:32:29.308926 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:32:29.308898 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837\": container with ID starting with 2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837 not found: ID does not exist" containerID="2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837" Jul 17 11:32:29.309012 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.308934 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837"} err="failed to get container status \"2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837\": rpc error: code = NotFound desc = could not find container \"2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837\": container with ID starting with 2064a409e85925d70aa28c0197f52f075983bab2beb157c1df391fb93ccbb837 not found: ID does not exist" Jul 17 11:32:29.313949 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:29.313923 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-71832244-openshift-default-6887bd4c9f-szswj"] Jul 17 11:32:30.823637 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:32:30.823604 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" path="/var/lib/kubelet/pods/b0b1a63c-6eb2-4962-8925-b185fdbcc7ea/volumes" Jul 17 11:34:00.645163 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:00.645117 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:34:10.948336 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:10.948252 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:34:23.750534 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:23.750492 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:34:29.782259 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:29.782219 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt"] Jul 17 11:34:29.782704 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:29.782585 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" Jul 17 11:34:29.782704 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:29.782597 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" Jul 17 11:34:29.782704 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:29.782673 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" Jul 17 11:34:29.785730 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:29.785713 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt" Jul 17 11:34:29.788483 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:29.788460 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 17 11:34:29.788483 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:29.788475 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87tmr\"" Jul 17 11:34:29.795162 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:29.795138 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt"] Jul 17 11:34:29.876889 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:29.876843 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4548058-73df-4462-97c6-735ab032cd1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ff1f9608-npntt\" (UID: \"e4548058-73df-4462-97c6-735ab032cd1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt" Jul 17 11:34:29.977808 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:29.977769 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4548058-73df-4462-97c6-735ab032cd1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ff1f9608-npntt\" (UID: \"e4548058-73df-4462-97c6-735ab032cd1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt" Jul 17 11:34:29.978483 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:29.978459 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4548058-73df-4462-97c6-735ab032cd1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ff1f9608-npntt\" (UID: \"e4548058-73df-4462-97c6-735ab032cd1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt" Jul 17 11:34:30.096247 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:30.096160 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt" Jul 17 11:34:30.237051 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:30.237025 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt"] Jul 17 11:34:30.239113 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:34:30.239081 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4548058_73df_4462_97c6_735ab032cd1a.slice/crio-3595177c4bf27b1090364c07e102e0af72f257a15229fc6fe008ca0bf35d53e5 WatchSource:0}: Error finding container 3595177c4bf27b1090364c07e102e0af72f257a15229fc6fe008ca0bf35d53e5: Status 404 returned error can't find the container with id 3595177c4bf27b1090364c07e102e0af72f257a15229fc6fe008ca0bf35d53e5 Jul 17 11:34:30.939734 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:30.939696 2551 generic.go:356] "Generic (PLEG): container finished" podID="e4548058-73df-4462-97c6-735ab032cd1a" containerID="4acd7470b4d84324b8ae774834daa42a6e73ddbfb054a6de115559aedaff400d" exitCode=0 Jul 17 11:34:30.940133 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:30.939758 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt" event={"ID":"e4548058-73df-4462-97c6-735ab032cd1a","Type":"ContainerDied","Data":"4acd7470b4d84324b8ae774834daa42a6e73ddbfb054a6de115559aedaff400d"} Jul 17 11:34:30.940133 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:30.939816 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt" event={"ID":"e4548058-73df-4462-97c6-735ab032cd1a","Type":"ContainerStarted","Data":"3595177c4bf27b1090364c07e102e0af72f257a15229fc6fe008ca0bf35d53e5"} Jul 17 11:34:32.080373 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:32.080348 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt" Jul 17 11:34:32.195189 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:32.195151 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4548058-73df-4462-97c6-735ab032cd1a-maas-api-service-ca\") pod \"e4548058-73df-4462-97c6-735ab032cd1a\" (UID: \"e4548058-73df-4462-97c6-735ab032cd1a\") " Jul 17 11:34:32.195575 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:32.195522 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4548058-73df-4462-97c6-735ab032cd1a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4548058-73df-4462-97c6-735ab032cd1a" (UID: "e4548058-73df-4462-97c6-735ab032cd1a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:34:32.296612 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:32.296577 2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4548058-73df-4462-97c6-735ab032cd1a-maas-api-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:34:32.952566 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:32.952511 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt" event={"ID":"e4548058-73df-4462-97c6-735ab032cd1a","Type":"ContainerDied","Data":"3595177c4bf27b1090364c07e102e0af72f257a15229fc6fe008ca0bf35d53e5"} Jul 17 11:34:32.952733 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:32.952587 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3595177c4bf27b1090364c07e102e0af72f257a15229fc6fe008ca0bf35d53e5" Jul 17 11:34:32.952733 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:32.952525 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff1f9608-npntt" Jul 17 11:34:43.548015 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:34:43.547979 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:35:25.143740 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:25.143702 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:35:34.633826 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:34.633777 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:35:42.034468 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:42.034379 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:35:46.145149 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.145111 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx"] Jul 17 11:35:46.145535 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.145481 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" Jul 17 11:35:46.145535 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.145493 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" Jul 17 11:35:46.145535 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.145502 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4548058-73df-4462-97c6-735ab032cd1a" containerName="revoke-keys" Jul 17 11:35:46.145535 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.145509 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4548058-73df-4462-97c6-735ab032cd1a" containerName="revoke-keys" Jul 17 11:35:46.145702 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.145595 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0b1a63c-6eb2-4962-8925-b185fdbcc7ea" containerName="istio-proxy" Jul 17 11:35:46.145702 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.145609 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4548058-73df-4462-97c6-735ab032cd1a" containerName="revoke-keys" Jul 17 11:35:46.148769 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.148743 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx" Jul 17 11:35:46.151204 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.151182 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87tmr\"" Jul 17 11:35:46.151320 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.151204 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 17 11:35:46.156116 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.156088 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx"] Jul 17 11:35:46.269388 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.269348 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2acee16-a567-4574-9c63-a9737f1dd5da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-caf75d03-svmhx\" (UID: \"c2acee16-a567-4574-9c63-a9737f1dd5da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx" Jul 17 11:35:46.370852 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.370820 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2acee16-a567-4574-9c63-a9737f1dd5da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-caf75d03-svmhx\" (UID: \"c2acee16-a567-4574-9c63-a9737f1dd5da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx" Jul 17 11:35:46.371478 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.371457 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2acee16-a567-4574-9c63-a9737f1dd5da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-caf75d03-svmhx\" (UID: \"c2acee16-a567-4574-9c63-a9737f1dd5da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx" Jul 17 11:35:46.461927 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.461838 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx" Jul 17 11:35:46.600775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:46.600744 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx"] Jul 17 11:35:46.602392 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:35:46.602359 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2acee16_a567_4574_9c63_a9737f1dd5da.slice/crio-86f76cea08875e4edfcbddf2f8803498bf0e9c0a5b71affefb7de0145531ad98 WatchSource:0}: Error finding container 86f76cea08875e4edfcbddf2f8803498bf0e9c0a5b71affefb7de0145531ad98: Status 404 returned error can't find the container with id 86f76cea08875e4edfcbddf2f8803498bf0e9c0a5b71affefb7de0145531ad98 Jul 17 11:35:47.351067 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:47.351032 2551 generic.go:356] "Generic (PLEG): container finished" podID="c2acee16-a567-4574-9c63-a9737f1dd5da" containerID="ecf75c9352ae69063e7f24bf7f8c1be294322aac48044d688f5c2deae227457a" exitCode=0 Jul 17 11:35:47.351434 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:47.351112 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx" event={"ID":"c2acee16-a567-4574-9c63-a9737f1dd5da","Type":"ContainerDied","Data":"ecf75c9352ae69063e7f24bf7f8c1be294322aac48044d688f5c2deae227457a"} Jul 17 11:35:47.351434 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:47.351165 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx" event={"ID":"c2acee16-a567-4574-9c63-a9737f1dd5da","Type":"ContainerStarted","Data":"86f76cea08875e4edfcbddf2f8803498bf0e9c0a5b71affefb7de0145531ad98"} Jul 17 11:35:48.495649 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:48.495625 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx" Jul 17 11:35:48.588417 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:48.588379 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2acee16-a567-4574-9c63-a9737f1dd5da-maas-api-service-ca\") pod \"c2acee16-a567-4574-9c63-a9737f1dd5da\" (UID: \"c2acee16-a567-4574-9c63-a9737f1dd5da\") " Jul 17 11:35:48.588813 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:48.588791 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2acee16-a567-4574-9c63-a9737f1dd5da-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c2acee16-a567-4574-9c63-a9737f1dd5da" (UID: "c2acee16-a567-4574-9c63-a9737f1dd5da"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:35:48.690062 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:48.689984 2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2acee16-a567-4574-9c63-a9737f1dd5da-maas-api-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:35:49.363427 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:49.363395 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx" Jul 17 11:35:49.363662 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:49.363440 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-caf75d03-svmhx" event={"ID":"c2acee16-a567-4574-9c63-a9737f1dd5da","Type":"ContainerDied","Data":"86f76cea08875e4edfcbddf2f8803498bf0e9c0a5b71affefb7de0145531ad98"} Jul 17 11:35:49.363662 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:35:49.363489 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86f76cea08875e4edfcbddf2f8803498bf0e9c0a5b71affefb7de0145531ad98" Jul 17 11:36:00.948298 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:36:00.948256 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:36:57.452438 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:36:57.452398 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:37:11.856185 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:11.856099 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:37:17.058900 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:17.058868 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:37:17.067832 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:17.067806 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:37:49.102612 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.102574 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq"] Jul 17 11:37:49.103131 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.102942 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2acee16-a567-4574-9c63-a9737f1dd5da" containerName="revoke-keys" Jul 17 11:37:49.103131 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.102953 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2acee16-a567-4574-9c63-a9737f1dd5da" containerName="revoke-keys" Jul 17 11:37:49.103131 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.103026 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2acee16-a567-4574-9c63-a9737f1dd5da" containerName="revoke-keys" Jul 17 11:37:49.106152 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.106130 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.108820 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.108795 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4da7d6-openshift-default-dockercfg-n79kh\"" Jul 17 11:37:49.121796 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.121767 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq"] Jul 17 11:37:49.146224 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.146195 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istiod-ca-cert\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.146366 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.146257 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-envoy\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.146366 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.146323 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-data\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.146366 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.146353 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-certs\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.146483 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.146394 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l96k\" (UniqueName: \"kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-kube-api-access-8l96k\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.146483 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.146451 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-socket\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.146483 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.146469 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-podinfo\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.146650 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.146487 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-credential-socket\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.146650 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.146506 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-token\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.247438 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247403 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-data\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.247438 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247441 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-certs\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.247697 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247472 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l96k\" (UniqueName: \"kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-kube-api-access-8l96k\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.247697 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247518 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-socket\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.247697 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247545 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-podinfo\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.247697 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247601 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-credential-socket\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.247697 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247622 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-token\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.247697 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247664 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istiod-ca-cert\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.248091 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247723 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-envoy\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.248091 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247874 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-data\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.248091 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247939 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-socket\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.248091 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.247973 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-credential-socket\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.248091 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.248011 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-certs\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.248521 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.248498 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istiod-ca-cert\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.249910 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.249888 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-envoy\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.250630 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.250609 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-podinfo\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.255673 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.255651 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-token\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.255924 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.255905 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l96k\" (UniqueName: \"kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-kube-api-access-8l96k\") pod \"e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.416017 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.415931 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:49.560343 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.560313 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq"] Jul 17 11:37:49.561979 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:37:49.561951 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33bb08cc_06cc_496c_90e2_5be9e551c8e7.slice/crio-fd6479929f43072324369bc532af24c9758b42d46f630ca9365831845e6b02ac WatchSource:0}: Error finding container fd6479929f43072324369bc532af24c9758b42d46f630ca9365831845e6b02ac: Status 404 returned error can't find the container with id fd6479929f43072324369bc532af24c9758b42d46f630ca9365831845e6b02ac Jul 17 11:37:49.564147 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.564112 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:37:49.564239 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.564192 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:37:49.564302 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:49.564240 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:37:50.019707 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:50.019669 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" event={"ID":"33bb08cc-06cc-496c-90e2-5be9e551c8e7","Type":"ContainerStarted","Data":"e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394"} Jul 17 11:37:50.019931 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:50.019728 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" event={"ID":"33bb08cc-06cc-496c-90e2-5be9e551c8e7","Type":"ContainerStarted","Data":"fd6479929f43072324369bc532af24c9758b42d46f630ca9365831845e6b02ac"} Jul 17 11:37:50.041976 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:50.041922 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" podStartSLOduration=1.041904985 podStartE2EDuration="1.041904985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:37:50.040923592 +0000 UTC m=+2733.714439245" watchObservedRunningTime="2026-07-17 11:37:50.041904985 +0000 UTC m=+2733.715420636" Jul 17 11:37:50.416671 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:50.416580 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:50.421946 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:50.421921 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:51.025152 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:51.025117 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:51.026035 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:51.026012 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:37:59.675981 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:37:59.675942 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:38:07.066767 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:07.066719 2551 patch_prober.go:28] interesting pod/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 17 11:38:07.067197 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:07.066788 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 17 11:38:08.131190 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:08.131156 2551 generic.go:356] "Generic (PLEG): container finished" podID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerID="e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394" exitCode=0 Jul 17 11:38:08.131658 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:08.131210 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" event={"ID":"33bb08cc-06cc-496c-90e2-5be9e551c8e7","Type":"ContainerDied","Data":"e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394"} Jul 17 11:38:08.131658 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:08.131611 2551 scope.go:117] "RemoveContainer" containerID="e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394" Jul 17 11:38:08.132787 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:08.132744 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:38:08.132884 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:08.132837 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:38:08.132884 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:08.132877 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:38:09.138188 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:09.138152 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" event={"ID":"33bb08cc-06cc-496c-90e2-5be9e551c8e7","Type":"ContainerStarted","Data":"09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52"} Jul 17 11:38:09.416799 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:09.416678 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:38:09.416992 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:09.416972 2551 patch_prober.go:28] interesting pod/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 17 11:38:09.417073 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:09.417032 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 17 11:38:10.416306 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:10.416270 2551 patch_prober.go:28] interesting pod/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 17 11:38:10.416803 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:10.416336 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 17 11:38:10.846246 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:10.846209 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:38:11.417080 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:11.417035 2551 patch_prober.go:28] interesting pod/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 17 11:38:11.417601 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:11.417104 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 17 11:38:12.416754 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:12.416718 2551 patch_prober.go:28] interesting pod/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 17 11:38:12.416975 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:12.416787 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 17 11:38:13.420638 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:13.420606 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:38:13.421129 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:13.420973 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:38:13.421650 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:13.421631 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:38:22.548729 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:22.548693 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:38:54.055888 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:38:54.055801 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:39:07.955967 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:07.955929 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:39:25.277995 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:25.277955 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq"] Jul 17 11:39:25.281807 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:25.281780 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq" Jul 17 11:39:25.284291 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:25.284267 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 17 11:39:25.284432 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:25.284336 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87tmr\"" Jul 17 11:39:25.288529 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:25.288503 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq"] Jul 17 11:39:25.419165 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:25.419128 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51719088-21f1-42ec-832a-aeb85a9d50df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq\" (UID: \"51719088-21f1-42ec-832a-aeb85a9d50df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq" Jul 17 11:39:25.520221 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:25.520180 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51719088-21f1-42ec-832a-aeb85a9d50df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq\" (UID: \"51719088-21f1-42ec-832a-aeb85a9d50df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq" Jul 17 11:39:25.520887 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:25.520866 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51719088-21f1-42ec-832a-aeb85a9d50df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq\" (UID: \"51719088-21f1-42ec-832a-aeb85a9d50df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq" Jul 17 11:39:25.596218 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:25.596122 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq" Jul 17 11:39:25.731317 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:25.731288 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq"] Jul 17 11:39:25.732589 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:39:25.732559 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51719088_21f1_42ec_832a_aeb85a9d50df.slice/crio-48a60a17cf942c365873c9fc312def060c31d56ef7de0303cadd43c102abcf30 WatchSource:0}: Error finding container 48a60a17cf942c365873c9fc312def060c31d56ef7de0303cadd43c102abcf30: Status 404 returned error can't find the container with id 48a60a17cf942c365873c9fc312def060c31d56ef7de0303cadd43c102abcf30 Jul 17 11:39:26.552623 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:26.552593 2551 generic.go:356] "Generic (PLEG): container finished" podID="51719088-21f1-42ec-832a-aeb85a9d50df" containerID="b741db7cb9354ab350ad132681d8d85fca65152736322ff496099297223f1471" exitCode=0 Jul 17 11:39:26.553010 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:26.552682 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq" event={"ID":"51719088-21f1-42ec-832a-aeb85a9d50df","Type":"ContainerDied","Data":"b741db7cb9354ab350ad132681d8d85fca65152736322ff496099297223f1471"} Jul 17 11:39:26.553010 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:26.552736 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq" event={"ID":"51719088-21f1-42ec-832a-aeb85a9d50df","Type":"ContainerStarted","Data":"48a60a17cf942c365873c9fc312def060c31d56ef7de0303cadd43c102abcf30"} Jul 17 11:39:27.701524 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:27.701499 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq" Jul 17 11:39:27.841988 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:27.841889 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51719088-21f1-42ec-832a-aeb85a9d50df-maas-api-service-ca\") pod \"51719088-21f1-42ec-832a-aeb85a9d50df\" (UID: \"51719088-21f1-42ec-832a-aeb85a9d50df\") " Jul 17 11:39:27.842301 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:27.842280 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51719088-21f1-42ec-832a-aeb85a9d50df-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "51719088-21f1-42ec-832a-aeb85a9d50df" (UID: "51719088-21f1-42ec-832a-aeb85a9d50df"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:39:27.942961 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:27.942911 2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51719088-21f1-42ec-832a-aeb85a9d50df-maas-api-service-ca\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:39:28.568216 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:28.568179 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq" event={"ID":"51719088-21f1-42ec-832a-aeb85a9d50df","Type":"ContainerDied","Data":"48a60a17cf942c365873c9fc312def060c31d56ef7de0303cadd43c102abcf30"} Jul 17 11:39:28.568216 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:28.568207 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq" Jul 17 11:39:28.568433 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:28.568237 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48a60a17cf942c365873c9fc312def060c31d56ef7de0303cadd43c102abcf30" Jul 17 11:39:28.848728 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:28.848647 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:39:45.938161 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:45.938125 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:39:51.091823 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:51.091782 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq"] Jul 17 11:39:51.092310 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:51.092039 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" containerID="cri-o://09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52" gracePeriod=30 Jul 17 11:39:56.346722 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.346696 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:39:56.392831 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.392742 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-socket\") pod \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " Jul 17 11:39:56.392831 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.392786 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-token\") pod \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " Jul 17 11:39:56.392831 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.392809 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-data\") pod \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " Jul 17 11:39:56.393095 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.392839 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-credential-socket\") pod \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " Jul 17 11:39:56.393095 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.392964 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istiod-ca-cert\") pod \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " Jul 17 11:39:56.393095 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393009 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-envoy\") pod \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " Jul 17 11:39:56.393095 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393032 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-certs\") pod \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " Jul 17 11:39:56.393095 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393070 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-podinfo\") pod \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " Jul 17 11:39:56.393362 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393082 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33bb08cc-06cc-496c-90e2-5be9e551c8e7" (UID: "33bb08cc-06cc-496c-90e2-5be9e551c8e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:39:56.393362 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393162 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l96k\" (UniqueName: \"kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-kube-api-access-8l96k\") pod \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\" (UID: \"33bb08cc-06cc-496c-90e2-5be9e551c8e7\") " Jul 17 11:39:56.393475 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393161 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33bb08cc-06cc-496c-90e2-5be9e551c8e7" (UID: "33bb08cc-06cc-496c-90e2-5be9e551c8e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:39:56.393475 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393299 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33bb08cc-06cc-496c-90e2-5be9e551c8e7" (UID: "33bb08cc-06cc-496c-90e2-5be9e551c8e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:39:56.393475 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393311 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33bb08cc-06cc-496c-90e2-5be9e551c8e7" (UID: "33bb08cc-06cc-496c-90e2-5be9e551c8e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:39:56.393676 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393511 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-socket\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:39:56.393676 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393524 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-credential-socket\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:39:56.393676 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393534 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istiod-ca-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:39:56.393676 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393573 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-workload-certs\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:39:56.393909 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.393883 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-data" (OuterVolumeSpecName: "istio-data") pod "33bb08cc-06cc-496c-90e2-5be9e551c8e7" (UID: "33bb08cc-06cc-496c-90e2-5be9e551c8e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:39:56.394996 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.394972 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-token" (OuterVolumeSpecName: "istio-token") pod "33bb08cc-06cc-496c-90e2-5be9e551c8e7" (UID: "33bb08cc-06cc-496c-90e2-5be9e551c8e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:39:56.395091 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.395048 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33bb08cc-06cc-496c-90e2-5be9e551c8e7" (UID: "33bb08cc-06cc-496c-90e2-5be9e551c8e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 11:39:56.395544 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.395516 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33bb08cc-06cc-496c-90e2-5be9e551c8e7" (UID: "33bb08cc-06cc-496c-90e2-5be9e551c8e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:39:56.395544 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.395544 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-kube-api-access-8l96k" (OuterVolumeSpecName: "kube-api-access-8l96k") pod "33bb08cc-06cc-496c-90e2-5be9e551c8e7" (UID: "33bb08cc-06cc-496c-90e2-5be9e551c8e7"). InnerVolumeSpecName "kube-api-access-8l96k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:39:56.494815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.494781 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-token\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:39:56.494815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.494810 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-data\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:39:56.494815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.494818 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-envoy\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:39:56.494815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.494825 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bb08cc-06cc-496c-90e2-5be9e551c8e7-istio-podinfo\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:39:56.495084 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.494834 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l96k\" (UniqueName: \"kubernetes.io/projected/33bb08cc-06cc-496c-90e2-5be9e551c8e7-kube-api-access-8l96k\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:39:56.732536 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.732446 2551 generic.go:356] "Generic (PLEG): container finished" podID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerID="09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52" exitCode=0 Jul 17 11:39:56.732536 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.732494 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" Jul 17 11:39:56.732736 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.732533 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" event={"ID":"33bb08cc-06cc-496c-90e2-5be9e551c8e7","Type":"ContainerDied","Data":"09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52"} Jul 17 11:39:56.732736 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.732600 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq" event={"ID":"33bb08cc-06cc-496c-90e2-5be9e551c8e7","Type":"ContainerDied","Data":"fd6479929f43072324369bc532af24c9758b42d46f630ca9365831845e6b02ac"} Jul 17 11:39:56.732736 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.732624 2551 scope.go:117] "RemoveContainer" containerID="09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52" Jul 17 11:39:56.742360 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.742342 2551 scope.go:117] "RemoveContainer" containerID="e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394" Jul 17 11:39:56.750538 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.750522 2551 scope.go:117] "RemoveContainer" containerID="09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52" Jul 17 11:39:56.750803 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:39:56.750784 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52\": container with ID starting with 09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52 not found: ID does not exist" containerID="09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52" Jul 17 11:39:56.750873 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.750810 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52"} err="failed to get container status \"09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52\": rpc error: code = NotFound desc = could not find container \"09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52\": container with ID starting with 09a3789fb2e6d21f3b6ebd187ce99681195239402c4629897218217d255ccf52 not found: ID does not exist" Jul 17 11:39:56.750873 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.750828 2551 scope.go:117] "RemoveContainer" containerID="e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394" Jul 17 11:39:56.751078 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:39:56.751061 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394\": container with ID starting with e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394 not found: ID does not exist" containerID="e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394" Jul 17 11:39:56.751118 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.751085 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394"} err="failed to get container status \"e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394\": rpc error: code = NotFound desc = could not find container \"e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394\": container with ID starting with e4506ced830cfeaad88fb0f9ffbcbc3054696e967ccf56ebca946792599d5394 not found: ID does not exist" Jul 17 11:39:56.759983 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.759960 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq"] Jul 17 11:39:56.764760 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.764739 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4da7d6-openshift-default-59d4769d45-wmtnq"] Jul 17 11:39:56.823976 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:39:56.823945 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" path="/var/lib/kubelet/pods/33bb08cc-06cc-496c-90e2-5be9e551c8e7/volumes" Jul 17 11:40:05.376883 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:05.376828 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:40:18.430702 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.430663 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99"] Jul 17 11:40:18.431098 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.431044 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" Jul 17 11:40:18.431098 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.431057 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" Jul 17 11:40:18.431098 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.431067 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" Jul 17 11:40:18.431098 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.431073 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" Jul 17 11:40:18.431098 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.431081 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51719088-21f1-42ec-832a-aeb85a9d50df" containerName="revoke-keys" Jul 17 11:40:18.431098 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.431089 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="51719088-21f1-42ec-832a-aeb85a9d50df" containerName="revoke-keys" Jul 17 11:40:18.431300 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.431144 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" Jul 17 11:40:18.431300 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.431154 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="51719088-21f1-42ec-832a-aeb85a9d50df" containerName="revoke-keys" Jul 17 11:40:18.435833 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.435815 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.439314 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.439293 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-02cb6e-openshift-default-dockercfg-4rxbd\"" Jul 17 11:40:18.477775 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.477743 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99"] Jul 17 11:40:18.599749 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.599709 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-envoy\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.599919 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.599759 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04cf2096-59fc-4e75-a19c-c701af86c439-istiod-ca-cert\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.599919 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.599792 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-istio-token\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.599997 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.599920 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-data\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.599997 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.599964 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-socket\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.599997 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.599983 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-certs\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.600092 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.600035 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-credential-socket\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.600092 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.600075 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04cf2096-59fc-4e75-a19c-c701af86c439-istio-podinfo\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.600162 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.600107 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72d2b\" (UniqueName: \"kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-kube-api-access-72d2b\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.700694 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.700599 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-data\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.700694 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.700655 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-socket\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.700907 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.700773 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-certs\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.700907 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.700825 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-credential-socket\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.700907 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.700861 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04cf2096-59fc-4e75-a19c-c701af86c439-istio-podinfo\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.701065 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.700908 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72d2b\" (UniqueName: \"kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-kube-api-access-72d2b\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.701065 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.700954 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-envoy\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.701065 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.700978 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04cf2096-59fc-4e75-a19c-c701af86c439-istiod-ca-cert\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.701065 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.701005 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-istio-token\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.701065 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.701019 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-socket\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.701298 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.701080 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-data\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.701298 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.701138 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-certs\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.701612 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.701590 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-credential-socket\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.701726 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.701707 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04cf2096-59fc-4e75-a19c-c701af86c439-istiod-ca-cert\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.703264 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.703239 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-envoy\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.703395 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.703377 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04cf2096-59fc-4e75-a19c-c701af86c439-istio-podinfo\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.730173 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.730147 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-istio-token\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.730300 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.730261 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72d2b\" (UniqueName: \"kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-kube-api-access-72d2b\") pod \"e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.745859 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.745834 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:18.927639 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.927613 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99"] Jul 17 11:40:18.929757 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:40:18.929731 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04cf2096_59fc_4e75_a19c_c701af86c439.slice/crio-e13dbc1eb6cf1d9cb6f5648dadddde0073dce63f40717376a6d51809f692e7e4 WatchSource:0}: Error finding container e13dbc1eb6cf1d9cb6f5648dadddde0073dce63f40717376a6d51809f692e7e4: Status 404 returned error can't find the container with id e13dbc1eb6cf1d9cb6f5648dadddde0073dce63f40717376a6d51809f692e7e4 Jul 17 11:40:18.932366 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.932335 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:40:18.932425 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.932406 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:40:18.932468 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:18.932439 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:40:19.861481 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:19.861445 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" event={"ID":"04cf2096-59fc-4e75-a19c-c701af86c439","Type":"ContainerStarted","Data":"1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13"} Jul 17 11:40:19.861894 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:19.861504 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" event={"ID":"04cf2096-59fc-4e75-a19c-c701af86c439","Type":"ContainerStarted","Data":"e13dbc1eb6cf1d9cb6f5648dadddde0073dce63f40717376a6d51809f692e7e4"} Jul 17 11:40:19.896391 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:19.896341 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" podStartSLOduration=1.896325786 podStartE2EDuration="1.896325786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:40:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:40:19.894770466 +0000 UTC m=+2883.568286118" watchObservedRunningTime="2026-07-17 11:40:19.896325786 +0000 UTC m=+2883.569841438" Jul 17 11:40:20.745980 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:20.745943 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:20.750988 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:20.750963 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:20.867720 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:20.867691 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:20.868862 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:20.868839 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:25.651042 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:25.651004 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:40:33.949897 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:33.949861 2551 generic.go:356] "Generic (PLEG): container finished" podID="04cf2096-59fc-4e75-a19c-c701af86c439" containerID="1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13" exitCode=0 Jul 17 11:40:33.950329 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:33.949932 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" event={"ID":"04cf2096-59fc-4e75-a19c-c701af86c439","Type":"ContainerDied","Data":"1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13"} Jul 17 11:40:33.950329 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:33.950324 2551 scope.go:117] "RemoveContainer" containerID="1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13" Jul 17 11:40:33.951325 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:33.951288 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:40:33.951457 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:33.951370 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:40:33.951457 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:33.951419 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:40:34.956566 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:34.956505 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" event={"ID":"04cf2096-59fc-4e75-a19c-c701af86c439","Type":"ContainerStarted","Data":"001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32"} Jul 17 11:40:35.747094 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:35.747048 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:35.748371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:35.748341 2551 patch_prober.go:28] interesting pod/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 17 11:40:35.748489 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:35.748404 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 17 11:40:35.867742 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:35.867708 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:36.746939 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:36.746896 2551 patch_prober.go:28] interesting pod/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 17 11:40:36.747426 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:36.746976 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 17 11:40:37.746670 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:37.746633 2551 patch_prober.go:28] interesting pod/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 17 11:40:37.746844 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:37.746696 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 17 11:40:38.358395 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:38.358357 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:40:38.746299 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:38.746198 2551 patch_prober.go:28] interesting pod/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 17 11:40:38.746299 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:38.746262 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 17 11:40:39.750936 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:39.750904 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:39.752181 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:39.752159 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:40:56.533945 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:40:56.533906 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:41:08.736814 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:08.736776 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99"] Jul 17 11:41:08.737338 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:08.737017 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" containerID="cri-o://001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32" gracePeriod=30 Jul 17 11:41:09.753963 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:09.753921 2551 patch_prober.go:28] interesting pod/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 11:41:09.754350 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:09.753996 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 11:41:13.993377 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:13.993352 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:41:14.102251 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102213 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04cf2096-59fc-4e75-a19c-c701af86c439-istio-podinfo\") pod \"04cf2096-59fc-4e75-a19c-c701af86c439\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " Jul 17 11:41:14.102456 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102273 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-certs\") pod \"04cf2096-59fc-4e75-a19c-c701af86c439\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " Jul 17 11:41:14.102456 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102307 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72d2b\" (UniqueName: \"kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-kube-api-access-72d2b\") pod \"04cf2096-59fc-4e75-a19c-c701af86c439\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " Jul 17 11:41:14.102456 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102336 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-credential-socket\") pod \"04cf2096-59fc-4e75-a19c-c701af86c439\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " Jul 17 11:41:14.102456 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102355 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-istio-token\") pod \"04cf2096-59fc-4e75-a19c-c701af86c439\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " Jul 17 11:41:14.102456 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102386 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-socket\") pod \"04cf2096-59fc-4e75-a19c-c701af86c439\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " Jul 17 11:41:14.102456 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102432 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-data\") pod \"04cf2096-59fc-4e75-a19c-c701af86c439\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " Jul 17 11:41:14.102816 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102468 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-envoy\") pod \"04cf2096-59fc-4e75-a19c-c701af86c439\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " Jul 17 11:41:14.102816 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102523 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04cf2096-59fc-4e75-a19c-c701af86c439-istiod-ca-cert\") pod \"04cf2096-59fc-4e75-a19c-c701af86c439\" (UID: \"04cf2096-59fc-4e75-a19c-c701af86c439\") " Jul 17 11:41:14.102816 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102687 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04cf2096-59fc-4e75-a19c-c701af86c439" (UID: "04cf2096-59fc-4e75-a19c-c701af86c439"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:41:14.102816 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102725 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04cf2096-59fc-4e75-a19c-c701af86c439" (UID: "04cf2096-59fc-4e75-a19c-c701af86c439"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:41:14.102816 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102775 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04cf2096-59fc-4e75-a19c-c701af86c439" (UID: "04cf2096-59fc-4e75-a19c-c701af86c439"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:41:14.103086 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102879 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-certs\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:41:14.103086 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102900 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-credential-socket\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:41:14.103086 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.102915 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-workload-socket\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:41:14.103086 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.103029 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04cf2096-59fc-4e75-a19c-c701af86c439-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04cf2096-59fc-4e75-a19c-c701af86c439" (UID: "04cf2096-59fc-4e75-a19c-c701af86c439"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:41:14.103581 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.103537 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-data" (OuterVolumeSpecName: "istio-data") pod "04cf2096-59fc-4e75-a19c-c701af86c439" (UID: "04cf2096-59fc-4e75-a19c-c701af86c439"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:41:14.104773 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.104735 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-istio-token" (OuterVolumeSpecName: "istio-token") pod "04cf2096-59fc-4e75-a19c-c701af86c439" (UID: "04cf2096-59fc-4e75-a19c-c701af86c439"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:41:14.104867 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.104839 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04cf2096-59fc-4e75-a19c-c701af86c439" (UID: "04cf2096-59fc-4e75-a19c-c701af86c439"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:41:14.104973 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.104958 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04cf2096-59fc-4e75-a19c-c701af86c439-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04cf2096-59fc-4e75-a19c-c701af86c439" (UID: "04cf2096-59fc-4e75-a19c-c701af86c439"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 11:41:14.105275 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.105247 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-kube-api-access-72d2b" (OuterVolumeSpecName: "kube-api-access-72d2b") pod "04cf2096-59fc-4e75-a19c-c701af86c439" (UID: "04cf2096-59fc-4e75-a19c-c701af86c439"). InnerVolumeSpecName "kube-api-access-72d2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:41:14.170005 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.169957 2551 generic.go:356] "Generic (PLEG): container finished" podID="04cf2096-59fc-4e75-a19c-c701af86c439" containerID="001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32" exitCode=0 Jul 17 11:41:14.170180 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.170015 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" Jul 17 11:41:14.170180 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.170050 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" event={"ID":"04cf2096-59fc-4e75-a19c-c701af86c439","Type":"ContainerDied","Data":"001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32"} Jul 17 11:41:14.170180 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.170107 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99" event={"ID":"04cf2096-59fc-4e75-a19c-c701af86c439","Type":"ContainerDied","Data":"e13dbc1eb6cf1d9cb6f5648dadddde0073dce63f40717376a6d51809f692e7e4"} Jul 17 11:41:14.170180 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.170130 2551 scope.go:117] "RemoveContainer" containerID="001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32" Jul 17 11:41:14.179619 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.179602 2551 scope.go:117] "RemoveContainer" containerID="1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13" Jul 17 11:41:14.187835 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.187817 2551 scope.go:117] "RemoveContainer" containerID="001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32" Jul 17 11:41:14.188091 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:41:14.188068 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32\": container with ID starting with 001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32 not found: ID does not exist" containerID="001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32" Jul 17 11:41:14.188144 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.188105 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32"} err="failed to get container status \"001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32\": rpc error: code = NotFound desc = could not find container \"001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32\": container with ID starting with 001a5d332df30342e90e3dfadd36741680f52220c74702cc9282662996f3aa32 not found: ID does not exist" Jul 17 11:41:14.188144 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.188124 2551 scope.go:117] "RemoveContainer" containerID="1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13" Jul 17 11:41:14.188340 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:41:14.188325 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13\": container with ID starting with 1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13 not found: ID does not exist" containerID="1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13" Jul 17 11:41:14.188377 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.188347 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13"} err="failed to get container status \"1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13\": rpc error: code = NotFound desc = could not find container \"1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13\": container with ID starting with 1049644d4a39a4a964ed7647c916c3a3c288f99d90282337de2273a456ec6a13 not found: ID does not exist" Jul 17 11:41:14.193861 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.193837 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99"] Jul 17 11:41:14.198591 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.198569 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-02cb6e-openshift-default-666dc44967-rfs99"] Jul 17 11:41:14.203346 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.203326 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-data\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:41:14.203346 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.203347 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04cf2096-59fc-4e75-a19c-c701af86c439-istio-envoy\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:41:14.203461 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.203356 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04cf2096-59fc-4e75-a19c-c701af86c439-istiod-ca-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:41:14.203461 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.203364 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04cf2096-59fc-4e75-a19c-c701af86c439-istio-podinfo\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:41:14.203461 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.203373 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72d2b\" (UniqueName: \"kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-kube-api-access-72d2b\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:41:14.203461 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.203389 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04cf2096-59fc-4e75-a19c-c701af86c439-istio-token\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:41:14.823341 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:14.823306 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" path="/var/lib/kubelet/pods/04cf2096-59fc-4e75-a19c-c701af86c439/volumes" Jul 17 11:41:22.847905 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:22.847863 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:41:35.908252 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.908215 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5"] Jul 17 11:41:35.908651 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.908612 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" Jul 17 11:41:35.908651 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.908624 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" Jul 17 11:41:35.908651 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.908642 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" Jul 17 11:41:35.908651 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.908648 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" Jul 17 11:41:35.908800 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.908703 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" Jul 17 11:41:35.908800 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.908713 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="04cf2096-59fc-4e75-a19c-c701af86c439" containerName="istio-proxy" Jul 17 11:41:35.908800 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.908720 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bb08cc-06cc-496c-90e2-5be9e551c8e7" containerName="istio-proxy" Jul 17 11:41:35.911989 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.911968 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:35.914602 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.914582 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-bbe1d4-openshift-default-dockercfg-h8cb4\"" Jul 17 11:41:35.924163 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.924141 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5"] Jul 17 11:41:35.988061 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.988025 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-data\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:35.988342 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.988314 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-certs\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:35.988540 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.988518 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ftdn\" (UniqueName: \"kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-kube-api-access-5ftdn\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:35.988792 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.988772 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49e2498c-2e23-4470-a284-df164f4935cc-istio-podinfo\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:35.989048 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.989028 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-socket\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:35.989215 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.989200 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-envoy\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:35.989376 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.989358 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-istio-token\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:35.989518 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.989502 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-credential-socket\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:35.989677 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:35.989650 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49e2498c-2e23-4470-a284-df164f4935cc-istiod-ca-cert\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.090618 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.090580 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-socket\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.090836 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.090636 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-envoy\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.090836 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.090673 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-istio-token\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.090836 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.090700 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-credential-socket\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.090836 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.090722 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49e2498c-2e23-4470-a284-df164f4935cc-istiod-ca-cert\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.090836 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.090800 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-data\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.090836 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.090824 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-certs\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.091120 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.090969 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ftdn\" (UniqueName: \"kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-kube-api-access-5ftdn\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.091120 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.091028 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49e2498c-2e23-4470-a284-df164f4935cc-istio-podinfo\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.091217 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.091114 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-credential-socket\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.091329 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.091288 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-data\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.091465 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.091360 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-certs\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.091570 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.091534 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49e2498c-2e23-4470-a284-df164f4935cc-istiod-ca-cert\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.091631 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.091566 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-socket\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.093063 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.093041 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-envoy\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.093827 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.093810 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49e2498c-2e23-4470-a284-df164f4935cc-istio-podinfo\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.098990 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.098971 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ftdn\" (UniqueName: \"kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-kube-api-access-5ftdn\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.099252 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.099231 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-istio-token\") pod \"e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.222715 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.222630 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:36.363458 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.363411 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5"] Jul 17 11:41:36.369783 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.369747 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:41:36.369921 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.369823 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:41:36.369921 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:36.369854 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:41:37.302727 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:37.302689 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" event={"ID":"49e2498c-2e23-4470-a284-df164f4935cc","Type":"ContainerStarted","Data":"47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2"} Jul 17 11:41:37.303122 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:37.302757 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" event={"ID":"49e2498c-2e23-4470-a284-df164f4935cc","Type":"ContainerStarted","Data":"98771590fcb18b5d77836a9f1cd4efc3b7fcb97c1064ee276cad469fd7e45edd"} Jul 17 11:41:37.322356 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:37.322309 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" podStartSLOduration=2.32229659 podStartE2EDuration="2.32229659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:41:37.321370805 +0000 UTC m=+2960.994886458" watchObservedRunningTime="2026-07-17 11:41:37.32229659 +0000 UTC m=+2960.995812242" Jul 17 11:41:38.223845 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:38.223803 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:38.228811 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:38.228783 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:38.308202 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:38.308170 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:38.309328 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:38.309304 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:43.342590 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:43.342482 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:41:51.380872 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:51.380837 2551 generic.go:356] "Generic (PLEG): container finished" podID="49e2498c-2e23-4470-a284-df164f4935cc" containerID="47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2" exitCode=0 Jul 17 11:41:51.381401 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:51.380916 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" event={"ID":"49e2498c-2e23-4470-a284-df164f4935cc","Type":"ContainerDied","Data":"47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2"} Jul 17 11:41:51.381475 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:51.381420 2551 scope.go:117] "RemoveContainer" containerID="47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2" Jul 17 11:41:51.382519 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:51.382490 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:41:51.382648 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:51.382571 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:41:51.382648 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:51.382603 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 17 11:41:52.388476 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:52.388441 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" event={"ID":"49e2498c-2e23-4470-a284-df164f4935cc","Type":"ContainerStarted","Data":"f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84"} Jul 17 11:41:53.223311 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:53.223269 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:53.224667 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:53.224626 2551 patch_prober.go:28] interesting pod/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 17 11:41:53.224813 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:53.224706 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" podUID="49e2498c-2e23-4470-a284-df164f4935cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 17 11:41:53.308768 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:53.308721 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:54.223173 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:54.223133 2551 patch_prober.go:28] interesting pod/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 17 11:41:54.223610 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:54.223205 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" podUID="49e2498c-2e23-4470-a284-df164f4935cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 17 11:41:55.223964 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:55.223928 2551 patch_prober.go:28] interesting pod/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 17 11:41:55.224379 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:55.223997 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" podUID="49e2498c-2e23-4470-a284-df164f4935cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 17 11:41:56.227303 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:56.227271 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:56.228283 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:56.228265 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:41:57.738479 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:41:57.738443 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:42:17.094881 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:42:17.094854 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:42:17.116188 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:42:17.116157 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:42:18.637534 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:42:18.637494 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:42:37.445117 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:42:37.445081 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:42:56.541985 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:42:56.541950 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:43:12.841264 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:43:12.841181 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:43:30.039029 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:43:30.038987 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:43:46.857567 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:43:46.857481 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:44:04.442816 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:04.442774 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:44:21.341729 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:21.341688 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:44:29.084437 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:29.084402 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xkn4j"] Jul 17 11:44:29.087820 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:29.087803 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" Jul 17 11:44:29.090665 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:29.090640 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6c2dh\"" Jul 17 11:44:29.111908 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:29.111874 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xkn4j"] Jul 17 11:44:29.194083 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:29.194047 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tfgb\" (UniqueName: \"kubernetes.io/projected/453220a8-6270-477b-bdf0-af1447b71a8f-kube-api-access-6tfgb\") pod \"kuadrant-operator-catalog-xkn4j\" (UID: \"453220a8-6270-477b-bdf0-af1447b71a8f\") " pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" Jul 17 11:44:29.295303 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:29.295263 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tfgb\" (UniqueName: \"kubernetes.io/projected/453220a8-6270-477b-bdf0-af1447b71a8f-kube-api-access-6tfgb\") pod \"kuadrant-operator-catalog-xkn4j\" (UID: \"453220a8-6270-477b-bdf0-af1447b71a8f\") " pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" Jul 17 11:44:29.304962 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:29.304935 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tfgb\" (UniqueName: \"kubernetes.io/projected/453220a8-6270-477b-bdf0-af1447b71a8f-kube-api-access-6tfgb\") pod \"kuadrant-operator-catalog-xkn4j\" (UID: \"453220a8-6270-477b-bdf0-af1447b71a8f\") " pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" Jul 17 11:44:29.397202 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:29.397113 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" Jul 17 11:44:29.536816 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:29.536790 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xkn4j"] Jul 17 11:44:29.539620 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:44:29.539584 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod453220a8_6270_477b_bdf0_af1447b71a8f.slice/crio-647d505052c37125b0e6d63ca873afd9afb5467297364867a960c3fb46d655d1 WatchSource:0}: Error finding container 647d505052c37125b0e6d63ca873afd9afb5467297364867a960c3fb46d655d1: Status 404 returned error can't find the container with id 647d505052c37125b0e6d63ca873afd9afb5467297364867a960c3fb46d655d1 Jul 17 11:44:29.540986 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:29.540967 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 11:44:30.274224 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:30.274191 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" event={"ID":"453220a8-6270-477b-bdf0-af1447b71a8f","Type":"ContainerStarted","Data":"647d505052c37125b0e6d63ca873afd9afb5467297364867a960c3fb46d655d1"} Jul 17 11:44:32.289323 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:32.289283 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" event={"ID":"453220a8-6270-477b-bdf0-af1447b71a8f","Type":"ContainerStarted","Data":"8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b"} Jul 17 11:44:32.306740 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:32.306675 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" podStartSLOduration=1.631539156 podStartE2EDuration="3.306659564s" totalImagesPullingTime="1.675120408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:44:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:44:32.304940778 +0000 UTC m=+3135.978456430" watchObservedRunningTime="2026-07-17 11:44:32.306659564 +0000 UTC m=+3135.980175215" Jul 17 11:44:39.247952 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:39.247917 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:44:39.397459 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:39.397364 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" Jul 17 11:44:39.397459 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:39.397444 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" Jul 17 11:44:39.420664 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:39.420636 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" Jul 17 11:44:40.354774 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:40.354746 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" Jul 17 11:44:40.393349 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:40.393314 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xkn4j"] Jul 17 11:44:42.343185 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:42.343144 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" podUID="453220a8-6270-477b-bdf0-af1447b71a8f" containerName="registry-server" containerID="cri-o://8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b" gracePeriod=2 Jul 17 11:44:42.598777 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:42.598704 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" Jul 17 11:44:42.699124 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:42.699087 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tfgb\" (UniqueName: \"kubernetes.io/projected/453220a8-6270-477b-bdf0-af1447b71a8f-kube-api-access-6tfgb\") pod \"453220a8-6270-477b-bdf0-af1447b71a8f\" (UID: \"453220a8-6270-477b-bdf0-af1447b71a8f\") " Jul 17 11:44:42.701240 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:42.701214 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/453220a8-6270-477b-bdf0-af1447b71a8f-kube-api-access-6tfgb" (OuterVolumeSpecName: "kube-api-access-6tfgb") pod "453220a8-6270-477b-bdf0-af1447b71a8f" (UID: "453220a8-6270-477b-bdf0-af1447b71a8f"). InnerVolumeSpecName "kube-api-access-6tfgb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:44:42.800368 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:42.800334 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tfgb\" (UniqueName: \"kubernetes.io/projected/453220a8-6270-477b-bdf0-af1447b71a8f-kube-api-access-6tfgb\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:44:43.349916 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:43.349883 2551 generic.go:356] "Generic (PLEG): container finished" podID="453220a8-6270-477b-bdf0-af1447b71a8f" containerID="8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b" exitCode=0 Jul 17 11:44:43.350321 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:43.349944 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" Jul 17 11:44:43.350321 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:43.349970 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" event={"ID":"453220a8-6270-477b-bdf0-af1447b71a8f","Type":"ContainerDied","Data":"8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b"} Jul 17 11:44:43.350321 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:43.350020 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xkn4j" event={"ID":"453220a8-6270-477b-bdf0-af1447b71a8f","Type":"ContainerDied","Data":"647d505052c37125b0e6d63ca873afd9afb5467297364867a960c3fb46d655d1"} Jul 17 11:44:43.350321 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:43.350041 2551 scope.go:117] "RemoveContainer" containerID="8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b" Jul 17 11:44:43.359871 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:43.359852 2551 scope.go:117] "RemoveContainer" containerID="8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b" Jul 17 11:44:43.360135 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:44:43.360107 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b\": container with ID starting with 8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b not found: ID does not exist" containerID="8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b" Jul 17 11:44:43.360190 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:43.360134 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b"} err="failed to get container status \"8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b\": rpc error: code = NotFound desc = could not find container \"8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b\": container with ID starting with 8c32f6be89c12bdac9756bf0aac4e738a4a4c90b8e9e488833a2210dea80957b not found: ID does not exist" Jul 17 11:44:43.369497 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:43.369473 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xkn4j"] Jul 17 11:44:43.371878 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:43.371859 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xkn4j"] Jul 17 11:44:44.823485 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:44.823440 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="453220a8-6270-477b-bdf0-af1447b71a8f" path="/var/lib/kubelet/pods/453220a8-6270-477b-bdf0-af1447b71a8f/volumes" Jul 17 11:44:55.145666 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:44:55.145621 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:45:00.137457 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.137418 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz"] Jul 17 11:45:00.137933 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.137864 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="453220a8-6270-477b-bdf0-af1447b71a8f" containerName="registry-server" Jul 17 11:45:00.137933 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.137877 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="453220a8-6270-477b-bdf0-af1447b71a8f" containerName="registry-server" Jul 17 11:45:00.138010 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.137968 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="453220a8-6270-477b-bdf0-af1447b71a8f" containerName="registry-server" Jul 17 11:45:00.140941 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.140922 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh"] Jul 17 11:45:00.141060 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.141042 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz" Jul 17 11:45:00.143558 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.143517 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-87tmr\"" Jul 17 11:45:00.144093 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.144076 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp"] Jul 17 11:45:00.144212 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.144194 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh" Jul 17 11:45:00.147312 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.147284 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp" Jul 17 11:45:00.150805 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.150750 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz"] Jul 17 11:45:00.153517 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.153492 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh"] Jul 17 11:45:00.154975 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.154954 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz" Jul 17 11:45:00.156215 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.156192 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp"] Jul 17 11:45:00.164376 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.164354 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh" Jul 17 11:45:00.199239 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.199214 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp" Jul 17 11:45:00.328073 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.327915 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz"] Jul 17 11:45:00.330401 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:45:00.330348 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c357c20_7f30_453c_86c4_867c205d6206.slice/crio-af61fdfcae99bc2b957213ea145b210fe0f86953bd318545b94fc47c5477dea8 WatchSource:0}: Error finding container af61fdfcae99bc2b957213ea145b210fe0f86953bd318545b94fc47c5477dea8: Status 404 returned error can't find the container with id af61fdfcae99bc2b957213ea145b210fe0f86953bd318545b94fc47c5477dea8 Jul 17 11:45:00.356401 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.356371 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh"] Jul 17 11:45:00.357606 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:45:00.357509 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49a45e4c_8a52_40ee_bb9a_4031d36f437c.slice/crio-62d445384c66039d7eceabad6c8d1f5c32189fd762a466bf8a4e9b564f516644 WatchSource:0}: Error finding container 62d445384c66039d7eceabad6c8d1f5c32189fd762a466bf8a4e9b564f516644: Status 404 returned error can't find the container with id 62d445384c66039d7eceabad6c8d1f5c32189fd762a466bf8a4e9b564f516644 Jul 17 11:45:00.381774 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.381752 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp"] Jul 17 11:45:00.387835 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:45:00.387771 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5542c25c_abce_4445_bffe_0f60a7e8b64d.slice/crio-e8b0e1ded6ab29a868cb170ded45e579538cbb36d2a942cfc510382f8a11dcbb WatchSource:0}: Error finding container e8b0e1ded6ab29a868cb170ded45e579538cbb36d2a942cfc510382f8a11dcbb: Status 404 returned error can't find the container with id e8b0e1ded6ab29a868cb170ded45e579538cbb36d2a942cfc510382f8a11dcbb Jul 17 11:45:00.444214 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.444174 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp" event={"ID":"5542c25c-abce-4445-bffe-0f60a7e8b64d","Type":"ContainerStarted","Data":"e8b0e1ded6ab29a868cb170ded45e579538cbb36d2a942cfc510382f8a11dcbb"} Jul 17 11:45:00.445342 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.445313 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh" event={"ID":"49a45e4c-8a52-40ee-bb9a-4031d36f437c","Type":"ContainerStarted","Data":"62d445384c66039d7eceabad6c8d1f5c32189fd762a466bf8a4e9b564f516644"} Jul 17 11:45:00.446329 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:00.446308 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz" event={"ID":"7c357c20-7f30-453c-86c4-867c205d6206","Type":"ContainerStarted","Data":"af61fdfcae99bc2b957213ea145b210fe0f86953bd318545b94fc47c5477dea8"} Jul 17 11:45:01.453773 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:01.453682 2551 generic.go:356] "Generic (PLEG): container finished" podID="5542c25c-abce-4445-bffe-0f60a7e8b64d" containerID="04196c62f2e4cba96ecc5a9294d68b498b4a144c2e38a20aa347ad3473ca8fb0" exitCode=0 Jul 17 11:45:01.454167 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:01.453763 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp" event={"ID":"5542c25c-abce-4445-bffe-0f60a7e8b64d","Type":"ContainerDied","Data":"04196c62f2e4cba96ecc5a9294d68b498b4a144c2e38a20aa347ad3473ca8fb0"} Jul 17 11:45:01.455371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:01.455339 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh" event={"ID":"49a45e4c-8a52-40ee-bb9a-4031d36f437c","Type":"ContainerStarted","Data":"0ceb45aa695618cdbfce766dbb2691b0e5587c6ac8796fdb8ca313c522553739"} Jul 17 11:45:01.456789 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:01.456759 2551 generic.go:356] "Generic (PLEG): container finished" podID="7c357c20-7f30-453c-86c4-867c205d6206" containerID="e0553fa49a87d10d2901ecd4fcc506e20ea32aa8ee83511089115ae306ffa3a8" exitCode=0 Jul 17 11:45:01.456913 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:01.456830 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz" event={"ID":"7c357c20-7f30-453c-86c4-867c205d6206","Type":"ContainerDied","Data":"e0553fa49a87d10d2901ecd4fcc506e20ea32aa8ee83511089115ae306ffa3a8"} Jul 17 11:45:01.506242 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:01.506198 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh" podStartSLOduration=1.506181325 podStartE2EDuration="1.506181325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:45:01.504519788 +0000 UTC m=+3165.178035441" watchObservedRunningTime="2026-07-17 11:45:01.506181325 +0000 UTC m=+3165.179696977" Jul 17 11:45:02.464111 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:02.464075 2551 generic.go:356] "Generic (PLEG): container finished" podID="49a45e4c-8a52-40ee-bb9a-4031d36f437c" containerID="0ceb45aa695618cdbfce766dbb2691b0e5587c6ac8796fdb8ca313c522553739" exitCode=0 Jul 17 11:45:02.464517 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:02.464158 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh" event={"ID":"49a45e4c-8a52-40ee-bb9a-4031d36f437c","Type":"ContainerDied","Data":"0ceb45aa695618cdbfce766dbb2691b0e5587c6ac8796fdb8ca313c522553739"} Jul 17 11:45:02.618415 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:02.618392 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp" Jul 17 11:45:02.621132 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:02.621111 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz" Jul 17 11:45:03.471703 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:03.471669 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz" Jul 17 11:45:03.472128 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:03.471706 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz" event={"ID":"7c357c20-7f30-453c-86c4-867c205d6206","Type":"ContainerDied","Data":"af61fdfcae99bc2b957213ea145b210fe0f86953bd318545b94fc47c5477dea8"} Jul 17 11:45:03.472128 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:03.471757 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af61fdfcae99bc2b957213ea145b210fe0f86953bd318545b94fc47c5477dea8" Jul 17 11:45:03.473106 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:03.473089 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp" Jul 17 11:45:03.473221 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:03.473131 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp" event={"ID":"5542c25c-abce-4445-bffe-0f60a7e8b64d","Type":"ContainerDied","Data":"e8b0e1ded6ab29a868cb170ded45e579538cbb36d2a942cfc510382f8a11dcbb"} Jul 17 11:45:03.473221 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:03.473170 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8b0e1ded6ab29a868cb170ded45e579538cbb36d2a942cfc510382f8a11dcbb" Jul 17 11:45:03.607695 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:03.607667 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh" Jul 17 11:45:04.480362 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:04.480336 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh" Jul 17 11:45:04.480362 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:04.480355 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738145-9zghh" event={"ID":"49a45e4c-8a52-40ee-bb9a-4031d36f437c","Type":"ContainerDied","Data":"62d445384c66039d7eceabad6c8d1f5c32189fd762a466bf8a4e9b564f516644"} Jul 17 11:45:04.480850 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:04.480397 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62d445384c66039d7eceabad6c8d1f5c32189fd762a466bf8a4e9b564f516644" Jul 17 11:45:12.349255 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:12.349214 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:45:28.846406 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:28.846370 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:45:47.162738 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:45:47.162702 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:46:06.444309 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:46:06.444274 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:46:26.042801 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:46:26.042717 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:46:46.645840 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:46:46.645801 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:47:02.235730 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:02.235687 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:47:13.686316 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.686276 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7400-140359784656944"] Jul 17 11:47:13.686815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.686672 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c357c20-7f30-453c-86c4-867c205d6206" containerName="cleanup" Jul 17 11:47:13.686815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.686684 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c357c20-7f30-453c-86c4-867c205d6206" containerName="cleanup" Jul 17 11:47:13.686815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.686710 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49a45e4c-8a52-40ee-bb9a-4031d36f437c" containerName="cleanup" Jul 17 11:47:13.686815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.686715 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="49a45e4c-8a52-40ee-bb9a-4031d36f437c" containerName="cleanup" Jul 17 11:47:13.686815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.686723 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5542c25c-abce-4445-bffe-0f60a7e8b64d" containerName="cleanup" Jul 17 11:47:13.686815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.686728 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5542c25c-abce-4445-bffe-0f60a7e8b64d" containerName="cleanup" Jul 17 11:47:13.686815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.686780 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5542c25c-abce-4445-bffe-0f60a7e8b64d" containerName="cleanup" Jul 17 11:47:13.686815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.686789 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c357c20-7f30-453c-86c4-867c205d6206" containerName="cleanup" Jul 17 11:47:13.686815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.686797 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="49a45e4c-8a52-40ee-bb9a-4031d36f437c" containerName="cleanup" Jul 17 11:47:13.689952 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.689936 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784656944" Jul 17 11:47:13.692649 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.692622 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-jdtk7\"" Jul 17 11:47:13.695653 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.695628 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7400-140359784656944"] Jul 17 11:47:13.748763 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.748729 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glxmr\" (UniqueName: \"kubernetes.io/projected/472ce9b3-61ee-4dc4-bea3-158c5da07025-kube-api-access-glxmr\") pod \"test-curl-7400-140359784656944\" (UID: \"472ce9b3-61ee-4dc4-bea3-158c5da07025\") " pod="opendatahub/test-curl-7400-140359784656944" Jul 17 11:47:13.850085 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.850044 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glxmr\" (UniqueName: \"kubernetes.io/projected/472ce9b3-61ee-4dc4-bea3-158c5da07025-kube-api-access-glxmr\") pod \"test-curl-7400-140359784656944\" (UID: \"472ce9b3-61ee-4dc4-bea3-158c5da07025\") " pod="opendatahub/test-curl-7400-140359784656944" Jul 17 11:47:13.858427 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:13.858397 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glxmr\" (UniqueName: \"kubernetes.io/projected/472ce9b3-61ee-4dc4-bea3-158c5da07025-kube-api-access-glxmr\") pod \"test-curl-7400-140359784656944\" (UID: \"472ce9b3-61ee-4dc4-bea3-158c5da07025\") " pod="opendatahub/test-curl-7400-140359784656944" Jul 17 11:47:14.000999 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:14.000973 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784656944" Jul 17 11:47:14.137683 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:14.137654 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7400-140359784656944"] Jul 17 11:47:14.139595 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:47:14.139538 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod472ce9b3_61ee_4dc4_bea3_158c5da07025.slice/crio-566c70db1cddb1424196de8d2639b8f015bba0efb34cb5e451809fc23b45fe46 WatchSource:0}: Error finding container 566c70db1cddb1424196de8d2639b8f015bba0efb34cb5e451809fc23b45fe46: Status 404 returned error can't find the container with id 566c70db1cddb1424196de8d2639b8f015bba0efb34cb5e451809fc23b45fe46 Jul 17 11:47:14.205231 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:14.205197 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7400-140359784656944" event={"ID":"472ce9b3-61ee-4dc4-bea3-158c5da07025","Type":"ContainerStarted","Data":"566c70db1cddb1424196de8d2639b8f015bba0efb34cb5e451809fc23b45fe46"} Jul 17 11:47:15.212502 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:15.212397 2551 generic.go:356] "Generic (PLEG): container finished" podID="472ce9b3-61ee-4dc4-bea3-158c5da07025" containerID="d747575a7f45081120b8af27a17b8f36ccd04e317866821b8003e1aee354223b" exitCode=0 Jul 17 11:47:15.212502 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:15.212442 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7400-140359784656944" event={"ID":"472ce9b3-61ee-4dc4-bea3-158c5da07025","Type":"ContainerDied","Data":"d747575a7f45081120b8af27a17b8f36ccd04e317866821b8003e1aee354223b"} Jul 17 11:47:15.342012 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:15.341976 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:47:16.352673 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.352644 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784656944" Jul 17 11:47:16.379601 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.379567 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glxmr\" (UniqueName: \"kubernetes.io/projected/472ce9b3-61ee-4dc4-bea3-158c5da07025-kube-api-access-glxmr\") pod \"472ce9b3-61ee-4dc4-bea3-158c5da07025\" (UID: \"472ce9b3-61ee-4dc4-bea3-158c5da07025\") " Jul 17 11:47:16.381952 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.381923 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/472ce9b3-61ee-4dc4-bea3-158c5da07025-kube-api-access-glxmr" (OuterVolumeSpecName: "kube-api-access-glxmr") pod "472ce9b3-61ee-4dc4-bea3-158c5da07025" (UID: "472ce9b3-61ee-4dc4-bea3-158c5da07025"). InnerVolumeSpecName "kube-api-access-glxmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:47:16.387595 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.387566 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7400-140359784656944_472ce9b3-61ee-4dc4-bea3-158c5da07025/test-curl-7400-140359784656944/0.log" Jul 17 11:47:16.411232 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.411201 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7400-140359784656944"] Jul 17 11:47:16.415362 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.415329 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7400-140359784656944"] Jul 17 11:47:16.480849 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.480745 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glxmr\" (UniqueName: \"kubernetes.io/projected/472ce9b3-61ee-4dc4-bea3-158c5da07025-kube-api-access-glxmr\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:16.587874 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.587833 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7400-140359784730320"] Jul 17 11:47:16.588302 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.588283 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="472ce9b3-61ee-4dc4-bea3-158c5da07025" containerName="test-curl-7400-140359784656944" Jul 17 11:47:16.588397 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.588304 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="472ce9b3-61ee-4dc4-bea3-158c5da07025" containerName="test-curl-7400-140359784656944" Jul 17 11:47:16.588397 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.588389 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="472ce9b3-61ee-4dc4-bea3-158c5da07025" containerName="test-curl-7400-140359784656944" Jul 17 11:47:16.591515 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.591494 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:16.598276 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.598248 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7400-140359784730320"] Jul 17 11:47:16.682289 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.682254 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdpzp\" (UniqueName: \"kubernetes.io/projected/40e1d983-0228-4745-8d27-33444f368235-kube-api-access-tdpzp\") pod \"test-curl-7400-140359784730320\" (UID: \"40e1d983-0228-4745-8d27-33444f368235\") " pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:16.783334 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.783290 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdpzp\" (UniqueName: \"kubernetes.io/projected/40e1d983-0228-4745-8d27-33444f368235-kube-api-access-tdpzp\") pod \"test-curl-7400-140359784730320\" (UID: \"40e1d983-0228-4745-8d27-33444f368235\") " pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:16.791858 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.791832 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdpzp\" (UniqueName: \"kubernetes.io/projected/40e1d983-0228-4745-8d27-33444f368235-kube-api-access-tdpzp\") pod \"test-curl-7400-140359784730320\" (UID: \"40e1d983-0228-4745-8d27-33444f368235\") " pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:16.824795 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.824753 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="472ce9b3-61ee-4dc4-bea3-158c5da07025" path="/var/lib/kubelet/pods/472ce9b3-61ee-4dc4-bea3-158c5da07025/volumes" Jul 17 11:47:16.902012 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:16.901970 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:17.042275 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:17.042248 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7400-140359784730320"] Jul 17 11:47:17.043071 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:47:17.043050 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40e1d983_0228_4745_8d27_33444f368235.slice/crio-24e5be5c1e8c7414f241f17ea5d997e8846e60e072fcf6c64827937f2991b750 WatchSource:0}: Error finding container 24e5be5c1e8c7414f241f17ea5d997e8846e60e072fcf6c64827937f2991b750: Status 404 returned error can't find the container with id 24e5be5c1e8c7414f241f17ea5d997e8846e60e072fcf6c64827937f2991b750 Jul 17 11:47:17.146658 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:17.146626 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:47:17.160347 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:17.160323 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:47:17.226106 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:17.226063 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7400-140359784730320" event={"ID":"40e1d983-0228-4745-8d27-33444f368235","Type":"ContainerStarted","Data":"24e5be5c1e8c7414f241f17ea5d997e8846e60e072fcf6c64827937f2991b750"} Jul 17 11:47:17.227365 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:17.227317 2551 scope.go:117] "RemoveContainer" containerID="d747575a7f45081120b8af27a17b8f36ccd04e317866821b8003e1aee354223b" Jul 17 11:47:17.227508 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:17.227346 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784656944" Jul 17 11:47:18.235618 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:18.235578 2551 generic.go:356] "Generic (PLEG): container finished" podID="40e1d983-0228-4745-8d27-33444f368235" containerID="17a9a32b98449885e62e2bd14b2fac6c70a728cc1fd52f31d43e206ee257952f" exitCode=0 Jul 17 11:47:18.236049 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:18.235660 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7400-140359784730320" event={"ID":"40e1d983-0228-4745-8d27-33444f368235","Type":"ContainerDied","Data":"17a9a32b98449885e62e2bd14b2fac6c70a728cc1fd52f31d43e206ee257952f"} Jul 17 11:47:19.379203 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.379177 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:19.401418 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.401390 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7400-140359784730320_40e1d983-0228-4745-8d27-33444f368235/test-curl-7400-140359784730320/0.log" Jul 17 11:47:19.408474 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.408451 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdpzp\" (UniqueName: \"kubernetes.io/projected/40e1d983-0228-4745-8d27-33444f368235-kube-api-access-tdpzp\") pod \"40e1d983-0228-4745-8d27-33444f368235\" (UID: \"40e1d983-0228-4745-8d27-33444f368235\") " Jul 17 11:47:19.410695 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.410665 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e1d983-0228-4745-8d27-33444f368235-kube-api-access-tdpzp" (OuterVolumeSpecName: "kube-api-access-tdpzp") pod "40e1d983-0228-4745-8d27-33444f368235" (UID: "40e1d983-0228-4745-8d27-33444f368235"). InnerVolumeSpecName "kube-api-access-tdpzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:47:19.423184 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.423147 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7400-140359784730320"] Jul 17 11:47:19.426825 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.426803 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7400-140359784730320"] Jul 17 11:47:19.509681 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.509649 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdpzp\" (UniqueName: \"kubernetes.io/projected/40e1d983-0228-4745-8d27-33444f368235-kube-api-access-tdpzp\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:19.554781 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.554746 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7400-140359784730320"] Jul 17 11:47:19.555135 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.555122 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e1d983-0228-4745-8d27-33444f368235" containerName="test-curl-7400-140359784730320" Jul 17 11:47:19.555180 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.555136 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e1d983-0228-4745-8d27-33444f368235" containerName="test-curl-7400-140359784730320" Jul 17 11:47:19.555221 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.555202 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e1d983-0228-4745-8d27-33444f368235" containerName="test-curl-7400-140359784730320" Jul 17 11:47:19.558334 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.558317 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:19.564828 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.564799 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7400-140359784730320"] Jul 17 11:47:19.610231 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.610192 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82txx\" (UniqueName: \"kubernetes.io/projected/8596bb91-315e-4a03-93d9-6cb391507d07-kube-api-access-82txx\") pod \"test-curl-7400-140359784730320\" (UID: \"8596bb91-315e-4a03-93d9-6cb391507d07\") " pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:19.711386 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.711351 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82txx\" (UniqueName: \"kubernetes.io/projected/8596bb91-315e-4a03-93d9-6cb391507d07-kube-api-access-82txx\") pod \"test-curl-7400-140359784730320\" (UID: \"8596bb91-315e-4a03-93d9-6cb391507d07\") " pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:19.719651 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.719613 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82txx\" (UniqueName: \"kubernetes.io/projected/8596bb91-315e-4a03-93d9-6cb391507d07-kube-api-access-82txx\") pod \"test-curl-7400-140359784730320\" (UID: \"8596bb91-315e-4a03-93d9-6cb391507d07\") " pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:19.869143 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:19.869046 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:20.006208 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:20.006182 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7400-140359784730320"] Jul 17 11:47:20.008140 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:47:20.008109 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8596bb91_315e_4a03_93d9_6cb391507d07.slice/crio-5f6fb9ccea8974ff115f722a9f7f94dfe76b518f5e0b157f99890186698de3ea WatchSource:0}: Error finding container 5f6fb9ccea8974ff115f722a9f7f94dfe76b518f5e0b157f99890186698de3ea: Status 404 returned error can't find the container with id 5f6fb9ccea8974ff115f722a9f7f94dfe76b518f5e0b157f99890186698de3ea Jul 17 11:47:20.249388 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:20.249285 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7400-140359784730320" event={"ID":"8596bb91-315e-4a03-93d9-6cb391507d07","Type":"ContainerStarted","Data":"5f6fb9ccea8974ff115f722a9f7f94dfe76b518f5e0b157f99890186698de3ea"} Jul 17 11:47:20.250609 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:20.250585 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24e5be5c1e8c7414f241f17ea5d997e8846e60e072fcf6c64827937f2991b750" Jul 17 11:47:20.250609 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:20.250589 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:20.262839 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:20.262807 2551 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7400-140359784730320" oldPodUID="40e1d983-0228-4745-8d27-33444f368235" podUID="8596bb91-315e-4a03-93d9-6cb391507d07" Jul 17 11:47:20.822783 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:20.822747 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40e1d983-0228-4745-8d27-33444f368235" path="/var/lib/kubelet/pods/40e1d983-0228-4745-8d27-33444f368235/volumes" Jul 17 11:47:21.257815 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:21.257779 2551 generic.go:356] "Generic (PLEG): container finished" podID="8596bb91-315e-4a03-93d9-6cb391507d07" containerID="002f3e81b760606f2b680026978337d1b65e4f26807d6565461ac5c4865d559c" exitCode=0 Jul 17 11:47:21.257987 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:21.257873 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7400-140359784730320" event={"ID":"8596bb91-315e-4a03-93d9-6cb391507d07","Type":"ContainerDied","Data":"002f3e81b760606f2b680026978337d1b65e4f26807d6565461ac5c4865d559c"} Jul 17 11:47:22.397422 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:22.397391 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:22.418741 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:22.418697 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7400-140359784730320_8596bb91-315e-4a03-93d9-6cb391507d07/test-curl-7400-140359784730320/0.log" Jul 17 11:47:22.437836 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:22.437805 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82txx\" (UniqueName: \"kubernetes.io/projected/8596bb91-315e-4a03-93d9-6cb391507d07-kube-api-access-82txx\") pod \"8596bb91-315e-4a03-93d9-6cb391507d07\" (UID: \"8596bb91-315e-4a03-93d9-6cb391507d07\") " Jul 17 11:47:22.439955 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:22.439928 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8596bb91-315e-4a03-93d9-6cb391507d07-kube-api-access-82txx" (OuterVolumeSpecName: "kube-api-access-82txx") pod "8596bb91-315e-4a03-93d9-6cb391507d07" (UID: "8596bb91-315e-4a03-93d9-6cb391507d07"). InnerVolumeSpecName "kube-api-access-82txx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:47:22.444764 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:22.444738 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7400-140359784730320"] Jul 17 11:47:22.448734 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:22.448710 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7400-140359784730320"] Jul 17 11:47:22.539455 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:22.539357 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82txx\" (UniqueName: \"kubernetes.io/projected/8596bb91-315e-4a03-93d9-6cb391507d07-kube-api-access-82txx\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:22.823246 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:22.823166 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8596bb91-315e-4a03-93d9-6cb391507d07" path="/var/lib/kubelet/pods/8596bb91-315e-4a03-93d9-6cb391507d07/volumes" Jul 17 11:47:23.270722 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:23.270691 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7400-140359784730320" Jul 17 11:47:23.270905 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:23.270740 2551 scope.go:117] "RemoveContainer" containerID="002f3e81b760606f2b680026978337d1b65e4f26807d6565461ac5c4865d559c" Jul 17 11:47:38.386425 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:38.386381 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp"] Jul 17 11:47:38.388957 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:38.388934 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bbe1d4-29738145-c76fp"] Jul 17 11:47:38.822348 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:38.822315 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5542c25c-abce-4445-bffe-0f60a7e8b64d" path="/var/lib/kubelet/pods/5542c25c-abce-4445-bffe-0f60a7e8b64d/volumes" Jul 17 11:47:41.750985 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:41.750937 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:47:53.652012 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:53.651975 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5"] Jul 17 11:47:53.652611 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:53.652277 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" podUID="49e2498c-2e23-4470-a284-df164f4935cc" containerName="istio-proxy" containerID="cri-o://f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84" gracePeriod=30 Jul 17 11:47:56.230568 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:56.230509 2551 patch_prober.go:28] interesting pod/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 11:47:56.230954 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:56.230596 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" podUID="49e2498c-2e23-4470-a284-df164f4935cc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 11:47:58.908452 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.908423 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:47:58.966605 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.966489 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49e2498c-2e23-4470-a284-df164f4935cc-istiod-ca-cert\") pod \"49e2498c-2e23-4470-a284-df164f4935cc\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " Jul 17 11:47:58.966605 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.966539 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-istio-token\") pod \"49e2498c-2e23-4470-a284-df164f4935cc\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " Jul 17 11:47:58.966605 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.966594 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-certs\") pod \"49e2498c-2e23-4470-a284-df164f4935cc\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " Jul 17 11:47:58.966857 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.966622 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49e2498c-2e23-4470-a284-df164f4935cc-istio-podinfo\") pod \"49e2498c-2e23-4470-a284-df164f4935cc\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " Jul 17 11:47:58.966857 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.966639 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-credential-socket\") pod \"49e2498c-2e23-4470-a284-df164f4935cc\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " Jul 17 11:47:58.966857 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.966684 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-data\") pod \"49e2498c-2e23-4470-a284-df164f4935cc\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " Jul 17 11:47:58.966857 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.966713 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ftdn\" (UniqueName: \"kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-kube-api-access-5ftdn\") pod \"49e2498c-2e23-4470-a284-df164f4935cc\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " Jul 17 11:47:58.966857 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.966737 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-socket\") pod \"49e2498c-2e23-4470-a284-df164f4935cc\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " Jul 17 11:47:58.966857 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.966818 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-envoy\") pod \"49e2498c-2e23-4470-a284-df164f4935cc\" (UID: \"49e2498c-2e23-4470-a284-df164f4935cc\") " Jul 17 11:47:58.967167 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.966893 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49e2498c-2e23-4470-a284-df164f4935cc" (UID: "49e2498c-2e23-4470-a284-df164f4935cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:47:58.967167 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.966946 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e2498c-2e23-4470-a284-df164f4935cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49e2498c-2e23-4470-a284-df164f4935cc" (UID: "49e2498c-2e23-4470-a284-df164f4935cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 11:47:58.967167 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.967063 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49e2498c-2e23-4470-a284-df164f4935cc" (UID: "49e2498c-2e23-4470-a284-df164f4935cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:47:58.967334 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.967212 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49e2498c-2e23-4470-a284-df164f4935cc-istiod-ca-cert\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:58.967334 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.967234 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-certs\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:58.967334 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.967252 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-credential-socket\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:58.967334 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.967214 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49e2498c-2e23-4470-a284-df164f4935cc" (UID: "49e2498c-2e23-4470-a284-df164f4935cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:47:58.967837 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.967806 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "49e2498c-2e23-4470-a284-df164f4935cc" (UID: "49e2498c-2e23-4470-a284-df164f4935cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:47:58.969059 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.969027 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-kube-api-access-5ftdn" (OuterVolumeSpecName: "kube-api-access-5ftdn") pod "49e2498c-2e23-4470-a284-df164f4935cc" (UID: "49e2498c-2e23-4470-a284-df164f4935cc"). InnerVolumeSpecName "kube-api-access-5ftdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:47:58.969059 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.969033 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49e2498c-2e23-4470-a284-df164f4935cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49e2498c-2e23-4470-a284-df164f4935cc" (UID: "49e2498c-2e23-4470-a284-df164f4935cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 11:47:58.969229 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.969214 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49e2498c-2e23-4470-a284-df164f4935cc" (UID: "49e2498c-2e23-4470-a284-df164f4935cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 11:47:58.969402 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:58.969386 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "49e2498c-2e23-4470-a284-df164f4935cc" (UID: "49e2498c-2e23-4470-a284-df164f4935cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 11:47:59.067926 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.067897 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-envoy\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:59.067926 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.067924 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-istio-token\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:59.067926 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.067933 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49e2498c-2e23-4470-a284-df164f4935cc-istio-podinfo\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:59.068154 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.067941 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-istio-data\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:59.068154 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.067949 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ftdn\" (UniqueName: \"kubernetes.io/projected/49e2498c-2e23-4470-a284-df164f4935cc-kube-api-access-5ftdn\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:59.068154 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.067958 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e2498c-2e23-4470-a284-df164f4935cc-workload-socket\") on node \"ip-10-0-143-42.ec2.internal\" DevicePath \"\"" Jul 17 11:47:59.478132 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.478094 2551 generic.go:356] "Generic (PLEG): container finished" podID="49e2498c-2e23-4470-a284-df164f4935cc" containerID="f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84" exitCode=0 Jul 17 11:47:59.478304 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.478162 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" Jul 17 11:47:59.478304 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.478179 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" event={"ID":"49e2498c-2e23-4470-a284-df164f4935cc","Type":"ContainerDied","Data":"f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84"} Jul 17 11:47:59.478304 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.478268 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5" event={"ID":"49e2498c-2e23-4470-a284-df164f4935cc","Type":"ContainerDied","Data":"98771590fcb18b5d77836a9f1cd4efc3b7fcb97c1064ee276cad469fd7e45edd"} Jul 17 11:47:59.478304 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.478292 2551 scope.go:117] "RemoveContainer" containerID="f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84" Jul 17 11:47:59.488015 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.487992 2551 scope.go:117] "RemoveContainer" containerID="47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2" Jul 17 11:47:59.496907 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.496888 2551 scope.go:117] "RemoveContainer" containerID="f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84" Jul 17 11:47:59.497153 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:47:59.497132 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84\": container with ID starting with f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84 not found: ID does not exist" containerID="f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84" Jul 17 11:47:59.497208 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.497162 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84"} err="failed to get container status \"f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84\": rpc error: code = NotFound desc = could not find container \"f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84\": container with ID starting with f1c3c36a48581bcbe6338ff99f99e47c585216abdd0613d786d2e96e0e759d84 not found: ID does not exist" Jul 17 11:47:59.497208 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.497178 2551 scope.go:117] "RemoveContainer" containerID="47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2" Jul 17 11:47:59.497406 ip-10-0-143-42 kubenswrapper[2551]: E0717 11:47:59.497389 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2\": container with ID starting with 47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2 not found: ID does not exist" containerID="47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2" Jul 17 11:47:59.497461 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.497417 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2"} err="failed to get container status \"47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2\": rpc error: code = NotFound desc = could not find container \"47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2\": container with ID starting with 47047517238d349e3262bd50f1c0308c7ea7258f4695bd74793340de6fb62aa2 not found: ID does not exist" Jul 17 11:47:59.501055 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.501035 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5"] Jul 17 11:47:59.507901 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:47:59.507881 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-bbe1d4-openshift-default-7cd78845b7-gvlk5"] Jul 17 11:48:00.823613 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:00.823569 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49e2498c-2e23-4470-a284-df164f4935cc" path="/var/lib/kubelet/pods/49e2498c-2e23-4470-a284-df164f4935cc/volumes" Jul 17 11:48:04.494255 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:04.494218 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz"] Jul 17 11:48:04.496723 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:04.496696 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7df590-29738145-w69xz"] Jul 17 11:48:04.823033 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:04.822999 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c357c20-7f30-453c-86c4-867c205d6206" path="/var/lib/kubelet/pods/7c357c20-7f30-453c-86c4-867c205d6206/volumes" Jul 17 11:48:07.636480 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:07.636440 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkz62"] Jul 17 11:48:25.926651 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:25.926609 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qjlhq_53ca2f01-0fa9-4a5f-9e1a-12303c1c0322/manager/0.log" Jul 17 11:48:26.051588 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:26.051533 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nsvgv_78425956-706c-4299-87fe-670db5f71eb9/manager/0.log" Jul 17 11:48:26.315163 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:26.315134 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fwnbx_d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d/server/0.log" Jul 17 11:48:26.547417 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:26.547387 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx_391cc4cd-934d-415c-b05f-f9a55d622937/copy-manifests/0.log" Jul 17 11:48:26.555392 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:26.555368 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx_391cc4cd-934d-415c-b05f-f9a55d622937/manager/0.log" Jul 17 11:48:27.515970 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:27.515938 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738145-9zghh_49a45e4c-8a52-40ee-bb9a-4031d36f437c/cleanup/0.log" Jul 17 11:48:27.855417 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:27.855328 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-4da7d6-cc6wq_51719088-21f1-42ec-832a-aeb85a9d50df/revoke-keys/0.log" Jul 17 11:48:28.071938 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:28.071908 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-caf75d03-svmhx_c2acee16-a567-4574-9c63-a9737f1dd5da/revoke-keys/0.log" Jul 17 11:48:28.181897 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:28.181812 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-ff1f9608-npntt_e4548058-73df-4462-97c6-735ab032cd1a/revoke-keys/0.log" Jul 17 11:48:29.313018 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.312983 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z_fc0649e1-69e8-46ae-8bf1-ae5442e11997/extract/0.log" Jul 17 11:48:29.320905 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.320875 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z_fc0649e1-69e8-46ae-8bf1-ae5442e11997/util/0.log" Jul 17 11:48:29.330682 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.330656 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z_fc0649e1-69e8-46ae-8bf1-ae5442e11997/pull/0.log" Jul 17 11:48:29.448152 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.448116 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj_34081b88-e909-4671-a7a4-93ced58d7c3a/util/0.log" Jul 17 11:48:29.455577 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.455536 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj_34081b88-e909-4671-a7a4-93ced58d7c3a/pull/0.log" Jul 17 11:48:29.462643 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.462624 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj_34081b88-e909-4671-a7a4-93ced58d7c3a/extract/0.log" Jul 17 11:48:29.573111 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.573031 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg_a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9/util/0.log" Jul 17 11:48:29.580303 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.580249 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg_a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9/pull/0.log" Jul 17 11:48:29.586930 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.586898 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg_a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9/extract/0.log" Jul 17 11:48:29.703992 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.703961 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz_5c6d33f7-0a85-4a6b-8723-6d1aefdc9689/util/0.log" Jul 17 11:48:29.710925 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.710897 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz_5c6d33f7-0a85-4a6b-8723-6d1aefdc9689/pull/0.log" Jul 17 11:48:29.718184 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:29.718159 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz_5c6d33f7-0a85-4a6b-8723-6d1aefdc9689/extract/0.log" Jul 17 11:48:30.572976 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:30.572947 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nkz62_34164918-cd9a-48fc-b616-2b718997ec7c/limitador/0.log" Jul 17 11:48:30.697155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:30.697120 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-sb6q8_0340f437-ce6f-4b84-bc7f-1fc107712c38/manager/0.log" Jul 17 11:48:31.181922 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:31.181893 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8xcdg_c6eedd6f-c67e-41c4-b5b6-36c25f71b139/discovery/0.log" Jul 17 11:48:31.296073 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:31.296044 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5445b9658d-7n5tn_c2c9ec8f-63ac-49a3-8b45-367c377cf692/kube-auth-proxy/0.log" Jul 17 11:48:31.532243 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:31.532203 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-56bm5_5aa62500-a5c0-4773-8aca-059227813d04/istio-proxy/1.log" Jul 17 11:48:32.366321 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:32.366290 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-btr4s_7842d7cd-0bde-47ae-a651-64efa592ae4e/main/0.log" Jul 17 11:48:41.199786 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:41.199748 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6m5q9_91a1ec59-ea70-461a-a9be-471315c4c7df/global-pull-secret-syncer/0.log" Jul 17 11:48:41.380141 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:41.380113 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ffbvd_339c2065-6a0f-4b69-b150-b18181ebb1f8/konnectivity-agent/0.log" Jul 17 11:48:41.498186 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:41.498096 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-42.ec2.internal_683047144508312a081996d7cfb41351/haproxy/0.log" Jul 17 11:48:46.132135 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.132096 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z_fc0649e1-69e8-46ae-8bf1-ae5442e11997/extract/0.log" Jul 17 11:48:46.159896 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.159868 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z_fc0649e1-69e8-46ae-8bf1-ae5442e11997/util/0.log" Jul 17 11:48:46.187259 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.187226 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596598z_fc0649e1-69e8-46ae-8bf1-ae5442e11997/pull/0.log" Jul 17 11:48:46.218991 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.218952 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj_34081b88-e909-4671-a7a4-93ced58d7c3a/extract/0.log" Jul 17 11:48:46.244723 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.244696 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj_34081b88-e909-4671-a7a4-93ced58d7c3a/util/0.log" Jul 17 11:48:46.272472 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.272441 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6kfj_34081b88-e909-4671-a7a4-93ced58d7c3a/pull/0.log" Jul 17 11:48:46.295230 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.295198 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg_a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9/extract/0.log" Jul 17 11:48:46.325694 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.325673 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg_a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9/util/0.log" Jul 17 11:48:46.350906 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.350883 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732bjbg_a1bc1939-5be6-4acc-9a77-ff0c82e9e9f9/pull/0.log" Jul 17 11:48:46.384532 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.384468 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz_5c6d33f7-0a85-4a6b-8723-6d1aefdc9689/extract/0.log" Jul 17 11:48:46.421428 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.421399 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz_5c6d33f7-0a85-4a6b-8723-6d1aefdc9689/util/0.log" Jul 17 11:48:46.453515 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:46.453489 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdtzz_5c6d33f7-0a85-4a6b-8723-6d1aefdc9689/pull/0.log" Jul 17 11:48:47.251317 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:47.251291 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nkz62_34164918-cd9a-48fc-b616-2b718997ec7c/limitador/0.log" Jul 17 11:48:47.335436 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:47.335402 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-sb6q8_0340f437-ce6f-4b84-bc7f-1fc107712c38/manager/0.log" Jul 17 11:48:49.361930 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:49.361897 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-pctt2_e34bda92-1aef-4cf9-9535-f3fdb9ed14ef/monitoring-plugin/0.log" Jul 17 11:48:49.395531 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:49.395502 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2fdjf_05445548-c931-4b7c-ac99-c8562fa1aa23/node-exporter/0.log" Jul 17 11:48:49.424664 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:49.424629 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2fdjf_05445548-c931-4b7c-ac99-c8562fa1aa23/kube-rbac-proxy/0.log" Jul 17 11:48:49.446817 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:49.446791 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2fdjf_05445548-c931-4b7c-ac99-c8562fa1aa23/init-textfile/0.log" Jul 17 11:48:49.924463 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:49.924405 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-2m52h_7f6b14f2-08d2-41a4-be04-511ac0c901ee/prometheus-operator/0.log" Jul 17 11:48:49.944035 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:49.944005 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-2m52h_7f6b14f2-08d2-41a4-be04-511ac0c901ee/kube-rbac-proxy/0.log" Jul 17 11:48:49.967915 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:49.967882 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-wxgr9_3345c573-10d3-433e-b9ee-42a5bfd6c3a1/prometheus-operator-admission-webhook/0.log" Jul 17 11:48:50.071788 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:50.071749 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78c87d5d66-86gsj_9ddf30d1-4651-443a-9ae8-d2bd0d8914a4/thanos-query/0.log" Jul 17 11:48:50.093875 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:50.093850 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78c87d5d66-86gsj_9ddf30d1-4651-443a-9ae8-d2bd0d8914a4/kube-rbac-proxy-web/0.log" Jul 17 11:48:50.113948 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:50.113905 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78c87d5d66-86gsj_9ddf30d1-4651-443a-9ae8-d2bd0d8914a4/kube-rbac-proxy/0.log" Jul 17 11:48:50.137371 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:50.137347 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78c87d5d66-86gsj_9ddf30d1-4651-443a-9ae8-d2bd0d8914a4/prom-label-proxy/0.log" Jul 17 11:48:50.158808 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:50.158785 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78c87d5d66-86gsj_9ddf30d1-4651-443a-9ae8-d2bd0d8914a4/kube-rbac-proxy-rules/0.log" Jul 17 11:48:50.184728 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:50.184665 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78c87d5d66-86gsj_9ddf30d1-4651-443a-9ae8-d2bd0d8914a4/kube-rbac-proxy-metrics/0.log" Jul 17 11:48:51.478027 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.477967 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr"] Jul 17 11:48:51.478407 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.478350 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e2498c-2e23-4470-a284-df164f4935cc" containerName="istio-proxy" Jul 17 11:48:51.478407 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.478361 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e2498c-2e23-4470-a284-df164f4935cc" containerName="istio-proxy" Jul 17 11:48:51.478407 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.478372 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e2498c-2e23-4470-a284-df164f4935cc" containerName="istio-proxy" Jul 17 11:48:51.478407 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.478378 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e2498c-2e23-4470-a284-df164f4935cc" containerName="istio-proxy" Jul 17 11:48:51.478407 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.478388 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8596bb91-315e-4a03-93d9-6cb391507d07" containerName="test-curl-7400-140359784730320" Jul 17 11:48:51.478407 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.478395 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8596bb91-315e-4a03-93d9-6cb391507d07" containerName="test-curl-7400-140359784730320" Jul 17 11:48:51.478660 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.478450 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8596bb91-315e-4a03-93d9-6cb391507d07" containerName="test-curl-7400-140359784730320" Jul 17 11:48:51.478660 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.478459 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e2498c-2e23-4470-a284-df164f4935cc" containerName="istio-proxy" Jul 17 11:48:51.478660 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.478465 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e2498c-2e23-4470-a284-df164f4935cc" containerName="istio-proxy" Jul 17 11:48:51.481822 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.481803 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.484780 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.484759 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m29nl\"/\"openshift-service-ca.crt\"" Jul 17 11:48:51.485757 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.485735 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m29nl\"/\"kube-root-ca.crt\"" Jul 17 11:48:51.485855 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.485757 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-m29nl\"/\"default-dockercfg-gjkbp\"" Jul 17 11:48:51.494138 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.494116 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr"] Jul 17 11:48:51.533156 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.533133 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-proc\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.533299 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.533166 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-sys\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.533299 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.533200 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-podres\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.533299 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.533221 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk7gc\" (UniqueName: \"kubernetes.io/projected/b2d63d8e-d448-455f-8cf8-8e5dd554e213-kube-api-access-zk7gc\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.533440 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.533319 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-lib-modules\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.634578 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.634513 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-lib-modules\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.634776 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.634611 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-proc\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.634776 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.634636 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-sys\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.634776 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.634676 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-podres\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.634776 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.634697 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-lib-modules\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.634776 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.634701 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk7gc\" (UniqueName: \"kubernetes.io/projected/b2d63d8e-d448-455f-8cf8-8e5dd554e213-kube-api-access-zk7gc\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.634776 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.634747 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-sys\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.634776 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.634745 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-proc\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.635022 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.634812 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2d63d8e-d448-455f-8cf8-8e5dd554e213-podres\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.645901 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.645872 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk7gc\" (UniqueName: \"kubernetes.io/projected/b2d63d8e-d448-455f-8cf8-8e5dd554e213-kube-api-access-zk7gc\") pod \"perf-node-gather-daemonset-vcgxr\" (UID: \"b2d63d8e-d448-455f-8cf8-8e5dd554e213\") " pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.791566 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.791513 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:51.924764 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:51.924732 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr"] Jul 17 11:48:51.926152 ip-10-0-143-42 kubenswrapper[2551]: W0717 11:48:51.926123 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb2d63d8e_d448_455f_8cf8_8e5dd554e213.slice/crio-1c73759fc6dcd9c88d7103a851667221c4c7e081fc53b2522f032ac141109c38 WatchSource:0}: Error finding container 1c73759fc6dcd9c88d7103a851667221c4c7e081fc53b2522f032ac141109c38: Status 404 returned error can't find the container with id 1c73759fc6dcd9c88d7103a851667221c4c7e081fc53b2522f032ac141109c38 Jul 17 11:48:52.020909 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:52.020886 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/1.log" Jul 17 11:48:52.028778 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:52.028756 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-r6gsp_55a709f2-51ec-4d79-923e-198fd980ba4b/console-operator/2.log" Jul 17 11:48:52.490835 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:52.490804 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-77458bd66c-xmkps_dd262627-495d-4948-894a-3924b0e292b3/console/0.log" Jul 17 11:48:52.777493 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:52.777462 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" event={"ID":"b2d63d8e-d448-455f-8cf8-8e5dd554e213","Type":"ContainerStarted","Data":"eafaa66fd4c6ab9fb7e9bb836248f289de6769ef45bfd5d5f1a3b81fd5cd792d"} Jul 17 11:48:52.777694 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:52.777514 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" event={"ID":"b2d63d8e-d448-455f-8cf8-8e5dd554e213","Type":"ContainerStarted","Data":"1c73759fc6dcd9c88d7103a851667221c4c7e081fc53b2522f032ac141109c38"} Jul 17 11:48:52.777694 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:52.777668 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:52.794917 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:52.794868 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" podStartSLOduration=1.79485022 podStartE2EDuration="1.79485022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 11:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 11:48:52.793475249 +0000 UTC m=+3396.466990902" watchObservedRunningTime="2026-07-17 11:48:52.79485022 +0000 UTC m=+3396.468365904" Jul 17 11:48:54.091872 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:54.091824 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-66wkj_46c254d9-45eb-404e-a08e-60c8bbf2314e/dns/0.log" Jul 17 11:48:54.113043 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:54.113009 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-66wkj_46c254d9-45eb-404e-a08e-60c8bbf2314e/kube-rbac-proxy/0.log" Jul 17 11:48:54.224414 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:54.224376 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bsl8q_5e2e9c83-f45d-44d7-b654-fd4187164f25/dns-node-resolver/0.log" Jul 17 11:48:54.835215 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:54.835183 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8qjkv_88f00775-a242-4da4-bfc9-c73a05c78e24/node-ca/0.log" Jul 17 11:48:56.007144 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:56.007109 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8xcdg_c6eedd6f-c67e-41c4-b5b6-36c25f71b139/discovery/0.log" Jul 17 11:48:56.027913 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:56.027885 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5445b9658d-7n5tn_c2c9ec8f-63ac-49a3-8b45-367c377cf692/kube-auth-proxy/0.log" Jul 17 11:48:56.086366 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:56.086335 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-56bm5_5aa62500-a5c0-4773-8aca-059227813d04/istio-proxy/0.log" Jul 17 11:48:56.151799 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:56.151768 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-56bm5_5aa62500-a5c0-4773-8aca-059227813d04/istio-proxy/1.log" Jul 17 11:48:56.970353 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:56.970322 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lw2nf_38ad94cb-fac6-47f0-9024-9d1a71f16caf/serve-healthcheck-canary/0.log" Jul 17 11:48:57.502303 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:57.502271 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-lxp5b_6c351f21-c45a-4d18-939d-5f4159aad6fb/insights-operator/0.log" Jul 17 11:48:57.506066 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:57.506039 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-lxp5b_6c351f21-c45a-4d18-939d-5f4159aad6fb/insights-operator/1.log" Jul 17 11:48:57.531074 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:57.531048 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2l2qk_357ff6cd-3d67-44d5-9f4d-638229f579a6/kube-rbac-proxy/0.log" Jul 17 11:48:57.554565 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:57.554528 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2l2qk_357ff6cd-3d67-44d5-9f4d-638229f579a6/exporter/0.log" Jul 17 11:48:57.577625 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:57.577600 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2l2qk_357ff6cd-3d67-44d5-9f4d-638229f579a6/extractor/0.log" Jul 17 11:48:58.794167 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:58.794137 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-m29nl/perf-node-gather-daemonset-vcgxr" Jul 17 11:48:59.825833 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:48:59.825795 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qjlhq_53ca2f01-0fa9-4a5f-9e1a-12303c1c0322/manager/0.log" Jul 17 11:49:00.053753 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:00.053726 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nsvgv_78425956-706c-4299-87fe-670db5f71eb9/manager/0.log" Jul 17 11:49:00.251055 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:00.251031 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fwnbx_d67fb8ad-a6ca-4d7e-a73e-a2862cb35e0d/server/0.log" Jul 17 11:49:00.365541 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:00.365506 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx_391cc4cd-934d-415c-b05f-f9a55d622937/manager/0.log" Jul 17 11:49:00.398862 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:00.398828 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f7bbb6f9-7dxbx_391cc4cd-934d-415c-b05f-f9a55d622937/copy-manifests/0.log" Jul 17 11:49:01.474155 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:01.474124 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8xcdg_c6eedd6f-c67e-41c4-b5b6-36c25f71b139/discovery/0.log" Jul 17 11:49:01.484368 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:01.484329 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5445b9658d-7n5tn_c2c9ec8f-63ac-49a3-8b45-367c377cf692/kube-auth-proxy/0.log" Jul 17 11:49:01.521261 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:01.521229 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-56bm5_5aa62500-a5c0-4773-8aca-059227813d04/istio-proxy/0.log" Jul 17 11:49:01.579898 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:01.579865 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-56bm5_5aa62500-a5c0-4773-8aca-059227813d04/istio-proxy/1.log" Jul 17 11:49:02.002397 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:02.002370 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wjtf2_2b5900ac-609d-42dd-a55e-9deb2b3053e6/openshift-lws-operator/0.log" Jul 17 11:49:03.003477 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:03.003448 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8xcdg_c6eedd6f-c67e-41c4-b5b6-36c25f71b139/discovery/0.log" Jul 17 11:49:03.017000 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:03.016978 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5445b9658d-7n5tn_c2c9ec8f-63ac-49a3-8b45-367c377cf692/kube-auth-proxy/0.log" Jul 17 11:49:03.060566 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:03.060508 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-56bm5_5aa62500-a5c0-4773-8aca-059227813d04/istio-proxy/0.log" Jul 17 11:49:03.129024 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:03.128988 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-56bm5_5aa62500-a5c0-4773-8aca-059227813d04/istio-proxy/1.log" Jul 17 11:49:08.197292 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:08.197260 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4j4j7_dff2a623-d2b3-4e48-830a-0d479108fa15/kube-multus-additional-cni-plugins/0.log" Jul 17 11:49:08.219507 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:08.219482 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4j4j7_dff2a623-d2b3-4e48-830a-0d479108fa15/egress-router-binary-copy/0.log" Jul 17 11:49:08.240130 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:08.240107 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4j4j7_dff2a623-d2b3-4e48-830a-0d479108fa15/cni-plugins/0.log" Jul 17 11:49:08.265159 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:08.265131 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4j4j7_dff2a623-d2b3-4e48-830a-0d479108fa15/bond-cni-plugin/0.log" Jul 17 11:49:08.285957 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:08.285928 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4j4j7_dff2a623-d2b3-4e48-830a-0d479108fa15/routeoverride-cni/0.log" Jul 17 11:49:08.306028 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:08.305998 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4j4j7_dff2a623-d2b3-4e48-830a-0d479108fa15/whereabouts-cni-bincopy/0.log" Jul 17 11:49:08.327925 ip-10-0-143-42 kubenswrapper[2551]: I0717 11:49:08.327893 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4j4j7_dff2a623-d2b3-4e48-830a-0d479108fa15/whereabouts-cni/0.log"