Jul 23 09:54:38.383896 ip-10-0-139-55 systemd[1]: Starting Kubernetes Kubelet... Jul 23 09:54:38.791023 ip-10-0-139-55 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 09:54:38.791023 ip-10-0-139-55 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 09:54:38.791023 ip-10-0-139-55 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 09:54:38.791023 ip-10-0-139-55 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 09:54:38.791023 ip-10-0-139-55 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 09:54:38.792290 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.792167 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 09:54:38.801854 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801822 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:54:38.801854 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801849 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:54:38.801854 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801853 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:54:38.801854 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801857 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:54:38.801854 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801860 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:54:38.801854 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801863 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:54:38.801854 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801866 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:54:38.801854 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801869 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801872 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801875 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801878 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801881 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801884 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801887 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801889 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801892 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801895 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801898 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801901 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801904 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801906 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801909 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801912 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801915 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801918 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801920 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801923 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:54:38.802158 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801925 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801928 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801930 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801932 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801935 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801937 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801939 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801942 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801945 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801947 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801950 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801952 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801955 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801957 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801960 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801964 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801966 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801969 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801972 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801974 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:54:38.802640 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801977 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801979 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801982 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801986 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801989 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801993 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.801997 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802000 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802003 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802006 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802009 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802011 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802014 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802016 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802019 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802022 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802024 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802027 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802029 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:54:38.803122 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802032 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802035 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802037 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802040 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802043 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802051 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802053 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802056 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802059 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802062 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802065 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802067 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802069 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802073 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802076 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802080 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802084 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802087 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802090 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:54:38.803634 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802092 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802095 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802097 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802100 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802102 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802106 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802110 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802113 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802116 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802119 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802121 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802124 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802127 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802130 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802883 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802890 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802894 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802897 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802900 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802904 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:54:38.804101 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802907 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802910 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802913 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802915 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802918 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802921 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802924 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802926 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802929 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802931 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802934 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802937 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802940 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802942 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802945 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802947 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802950 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802952 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802955 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802957 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:54:38.804587 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802960 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802962 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802965 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802968 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802970 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802973 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802975 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802978 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802980 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802983 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802985 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802990 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802994 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802997 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.802999 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803003 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803006 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803010 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803012 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:54:38.805080 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803015 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803018 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803021 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803024 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803027 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803030 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803033 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803035 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803038 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803041 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803043 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803045 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803048 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803050 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803053 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803056 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803059 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803062 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803064 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803067 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:54:38.805537 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803069 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803072 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803074 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803077 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803080 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803083 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803086 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803088 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803091 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803093 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803097 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803099 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803102 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803104 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803107 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803110 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803112 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803115 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803118 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803121 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:54:38.806042 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803124 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803126 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803129 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803131 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803134 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803137 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803139 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803142 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803144 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803147 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803149 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803152 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803154 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803156 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803597 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803604 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803608 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803611 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803614 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803618 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:54:38.806531 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803621 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803623 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803626 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803629 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803632 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803635 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803647 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803652 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803656 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803659 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803662 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803665 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803668 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803671 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803673 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803676 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803679 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803681 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803684 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803687 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:54:38.807039 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803689 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803692 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803694 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803697 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803699 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803702 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803705 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803710 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803713 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803715 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803718 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803720 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803723 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803726 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803728 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803730 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803733 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803735 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803738 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:54:38.807526 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803741 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803743 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803746 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803748 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803751 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803755 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803759 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803762 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803765 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803767 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803770 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803772 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803775 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803777 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803780 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803782 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803785 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803787 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803790 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803792 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:54:38.808012 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803813 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803819 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803823 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803826 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803828 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803831 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803834 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803837 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803841 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803844 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803847 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803850 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803855 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803858 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803861 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803864 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803866 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803869 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803872 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:54:38.808502 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803875 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803877 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803880 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803882 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803885 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803887 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803890 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803893 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803895 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803898 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803900 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803903 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803905 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803909 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.803913 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.803985 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.803993 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804001 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804005 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804009 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804012 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 09:54:38.808977 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804017 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804021 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804024 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804027 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804030 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804033 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804036 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804039 2580 flags.go:64] FLAG: --cgroup-root="" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804042 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804045 2580 flags.go:64] FLAG: --client-ca-file="" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804048 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804051 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804060 2580 flags.go:64] FLAG: --cluster-domain="" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804062 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804065 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804068 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804071 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804075 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804078 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804081 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804084 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804087 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804090 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804092 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 09:54:38.809496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804097 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804101 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804105 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804108 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804111 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804114 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804118 2580 flags.go:64] FLAG: --enable-server="true" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804121 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804125 2580 flags.go:64] FLAG: --event-burst="100" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804129 2580 flags.go:64] FLAG: --event-qps="50" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804131 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804134 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804137 2580 flags.go:64] FLAG: --eviction-hard="" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804141 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804143 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804146 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804149 2580 flags.go:64] FLAG: --eviction-soft="" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804152 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804155 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804158 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804161 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804163 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804166 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804169 2580 flags.go:64] FLAG: --feature-gates="" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804173 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 09:54:38.810165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804175 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804178 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804181 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804184 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804187 2580 flags.go:64] FLAG: --help="false" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804190 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-139-55.ec2.internal" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804193 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804197 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804201 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804205 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804208 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804210 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804213 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804216 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804218 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804221 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804224 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804227 2580 flags.go:64] FLAG: --kube-reserved="" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804230 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804232 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804235 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804238 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804241 2580 flags.go:64] FLAG: --lock-file="" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804243 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 09:54:38.810797 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804247 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804249 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804255 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804258 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804261 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804263 2580 flags.go:64] FLAG: --logging-format="text" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804266 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804269 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804272 2580 flags.go:64] FLAG: --manifest-url="" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804275 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804279 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804282 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804286 2580 flags.go:64] FLAG: --max-pods="110" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804288 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804291 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804295 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804300 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804303 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804306 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804308 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804316 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804319 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804322 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 09:54:38.811436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804325 2580 flags.go:64] FLAG: --pod-cidr="" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804327 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804333 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804336 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804339 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804342 2580 flags.go:64] FLAG: --port="10250" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804345 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804348 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b7a7f9e8c57876ef" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804351 2580 flags.go:64] FLAG: --qos-reserved="" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804354 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804357 2580 flags.go:64] FLAG: --register-node="true" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804359 2580 flags.go:64] FLAG: --register-with-taints="" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804363 2580 flags.go:64] FLAG: --registry-burst="10" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804366 2580 flags.go:64] FLAG: --registry-qps="5" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804369 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804372 2580 flags.go:64] FLAG: --reserved-memory="" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804375 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804378 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804381 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804383 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804386 2580 flags.go:64] FLAG: --runonce="false" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804389 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804391 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804394 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804397 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 09:54:38.812488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804402 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804405 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804408 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804411 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804413 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804416 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804419 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804421 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804424 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804427 2580 flags.go:64] FLAG: --system-cgroups="" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804430 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804435 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804438 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804441 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804445 2580 flags.go:64] FLAG: --tls-min-version="" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804447 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804450 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804453 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804456 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804458 2580 flags.go:64] FLAG: --v="2" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804462 2580 flags.go:64] FLAG: --version="false" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804467 2580 flags.go:64] FLAG: --vmodule="" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804470 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.804473 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804607 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:54:38.813549 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804612 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804615 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804618 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804621 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804624 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804627 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804630 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804635 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804638 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804640 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804643 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804646 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804648 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804651 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804654 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804657 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804659 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804662 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804664 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:54:38.814266 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804668 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804672 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804675 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804678 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804681 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804685 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804688 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804692 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804695 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804697 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804700 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804703 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804706 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804708 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804711 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804713 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804716 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804718 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804721 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804723 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:54:38.815096 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804730 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804733 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804735 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804738 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804741 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804743 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804746 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804749 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804752 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804754 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804756 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804759 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804761 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804764 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804766 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804769 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804771 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804774 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804776 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804779 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:54:38.815960 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804782 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804784 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804787 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804794 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804797 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804799 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804802 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804804 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804807 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804809 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804812 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804814 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804819 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804821 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804824 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804827 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804829 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804831 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804834 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:54:38.816589 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804837 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804839 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804842 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804845 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804847 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804850 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804852 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804855 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804857 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804860 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804862 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804865 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804867 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804869 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804872 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804874 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804878 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804882 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804886 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:54:38.817367 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.804889 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.805764 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.813693 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.813717 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.814403 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.814417 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814519 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814528 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814533 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814538 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814542 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814547 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814551 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814555 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814578 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:54:38.818139 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814583 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814586 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814590 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814593 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814597 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814601 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814605 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814610 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814614 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814618 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814622 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814626 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814630 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814634 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814638 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814642 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814645 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814651 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814658 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814663 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:54:38.818638 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814669 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814674 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814678 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814682 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814689 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814695 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814700 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814705 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814710 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814714 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814719 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814723 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814728 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814732 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814736 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814740 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814744 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814749 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814753 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:54:38.819177 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814757 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814762 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814766 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814771 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814775 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814779 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814783 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814787 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814791 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814795 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814799 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814804 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814809 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814813 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814818 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814822 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814826 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814831 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814836 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814840 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:54:38.819675 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814844 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814848 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814853 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814857 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814861 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814866 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814870 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814874 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814881 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814887 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814891 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814895 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814899 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814904 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814908 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814912 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814916 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814921 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814925 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:54:38.820214 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814929 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814933 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814937 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814942 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814947 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814951 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814956 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814960 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814964 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814969 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814973 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.814977 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.814986 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815185 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815195 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:54:38.820747 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815201 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815206 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815211 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815215 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815219 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815224 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815228 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815232 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815236 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815241 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815245 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815249 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815254 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815258 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815262 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815267 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815271 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815276 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815280 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815284 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:54:38.821126 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815288 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815292 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815297 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815302 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815306 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815310 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815316 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815320 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815324 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815329 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815333 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815337 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815342 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815346 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815351 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815355 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815360 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815364 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815369 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:54:38.821671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815373 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815377 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815381 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815385 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815389 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815394 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815398 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815402 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815406 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815410 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815414 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815418 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815422 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815427 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815433 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815438 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815443 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815447 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815451 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815455 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:54:38.822241 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815460 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815464 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815469 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815473 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815477 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815481 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815488 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815493 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815498 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815502 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815507 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815511 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815516 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815520 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815525 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815529 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815533 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815537 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815541 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:54:38.823062 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815546 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815552 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815577 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815583 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815587 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815591 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815596 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815600 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815605 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815609 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815613 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815617 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815621 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815625 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815630 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815635 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815648 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815653 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:54:38.823893 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:38.815657 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:54:38.824497 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.815665 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 09:54:38.824497 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.815859 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 09:54:38.824497 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.818447 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 09:54:38.824497 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.819868 2580 server.go:1019] "Starting client certificate rotation" Jul 23 09:54:38.824497 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.819978 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 09:54:38.824497 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.820022 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 09:54:38.824497 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.822621 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 09:54:38.825349 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.825329 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 09:54:38.836745 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.836719 2580 log.go:25] "Validated CRI v1 runtime API" Jul 23 09:54:38.838775 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.838759 2580 log.go:25] "Validated CRI v1 image API" Jul 23 09:54:38.840345 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.840315 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 09:54:38.842890 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.842868 2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ac4c7815-69c7-482e-8f39-61229cdcf158:/dev/nvme0n1p3 b197480b-9b34-49df-ba63-8fd75b9ba445:/dev/nvme0n1p4] Jul 23 09:54:38.842971 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.842890 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 09:54:38.844472 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.844448 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 09:54:38.848870 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.848745 2580 manager.go:216] Machine: {Timestamp:2026-07-23 09:54:38.847406045 +0000 UTC m=+0.346991799 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098248 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f4fbeb9a0aa1f7ea7be48413cff24 SystemUUID:ec2f4fbe-b9a0-aa1f-7ea7-be48413cff24 BootID:190a6c49-fd6b-47c1-ba1b-70d9191734d3 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b4:b3:97:f5:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b4:b3:97:f5:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:54:0f:50:c9:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 09:54:38.848870 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.848860 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 09:54:38.849007 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.848950 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 09:54:38.852164 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.852128 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 09:54:38.852311 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.852166 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-55.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 09:54:38.852859 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.852846 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 09:54:38.852892 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.852861 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 09:54:38.852892 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.852869 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 09:54:38.853703 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.853690 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 09:54:38.855077 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.855062 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 09:54:38.857874 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.857859 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 23 09:54:38.858010 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.858000 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 09:54:38.858044 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.858033 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 23 09:54:38.858044 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.858042 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 09:54:38.858097 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.858054 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 09:54:38.858097 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.858065 2580 kubelet.go:398] "Adding apiserver pod source" Jul 23 09:54:38.858097 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.858074 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 09:54:38.858503 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.858488 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 09:54:38.858573 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.858508 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 09:54:38.860903 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.860881 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 09:54:38.862325 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.862308 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 09:54:38.862432 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.862346 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 09:54:38.863526 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863505 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 09:54:38.863603 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863530 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 09:54:38.863603 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863537 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 09:54:38.863603 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863542 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 09:54:38.863603 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863551 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 09:54:38.863603 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863573 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 09:54:38.863603 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863582 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 09:54:38.863603 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863587 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 09:54:38.863603 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863598 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 09:54:38.863603 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863604 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 09:54:38.863843 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863613 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 09:54:38.863843 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.863619 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 09:54:38.864812 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.864797 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b7sp7" Jul 23 09:54:38.865087 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.865076 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 09:54:38.865120 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.865089 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 09:54:38.868960 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.868942 2580 server.go:1268] "Started kubelet" Jul 23 09:54:38.869714 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.869582 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 09:54:38.870543 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.869780 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 09:54:38.870543 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.869825 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 09:54:38.869841 ip-10-0-139-55 systemd[1]: Started Kubernetes Kubelet. Jul 23 09:54:38.873241 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.873214 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b7sp7" Jul 23 09:54:38.874685 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.874645 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-55.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 09:54:38.874957 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:38.874927 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 09:54:38.875011 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:38.874919 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 09:54:38.875088 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.875072 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 23 09:54:38.875845 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.875829 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 09:54:38.875933 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:38.874964 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-55.ec2.internal.18c4e29449edacfb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-55.ec2.internal,UID:ip-10-0-139-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-55.ec2.internal,},FirstTimestamp:2026-07-23 09:54:38.868909307 +0000 UTC m=+0.368495061,LastTimestamp:2026-07-23 09:54:38.868909307 +0000 UTC m=+0.368495061,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-55.ec2.internal,}" Jul 23 09:54:38.876264 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:38.876242 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 09:54:38.882616 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.882592 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 09:54:38.882616 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.882586 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 09:54:38.882836 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:38.882794 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:38.883380 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.883353 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 09:54:38.883380 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.883381 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 09:54:38.883743 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.883513 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 23 09:54:38.883743 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.883523 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 23 09:54:38.883743 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.883542 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 09:54:38.883915 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.883808 2580 factory.go:55] Registering systemd factory Jul 23 09:54:38.883915 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.883825 2580 factory.go:223] Registration of the systemd container factory successfully Jul 23 09:54:38.884145 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.884065 2580 factory.go:153] Registering CRI-O factory Jul 23 09:54:38.884145 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.884084 2580 factory.go:223] Registration of the crio container factory successfully Jul 23 09:54:38.884277 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.884147 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 09:54:38.884277 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.884173 2580 factory.go:103] Registering Raw factory Jul 23 09:54:38.884277 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.884187 2580 manager.go:1196] Started watching for new ooms in manager Jul 23 09:54:38.884825 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:38.884805 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 09:54:38.884902 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:38.884877 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 09:54:38.885417 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.885402 2580 manager.go:349] Starting recovery of all containers Jul 23 09:54:38.891897 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.891853 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 09:54:38.898057 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.898036 2580 manager.go:354] Recovery completed Jul 23 09:54:38.902474 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.902459 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:54:38.904789 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.904766 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:54:38.904878 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.904795 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:54:38.904878 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.904805 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:54:38.905357 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.905341 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 09:54:38.905357 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.905354 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 09:54:38.905491 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.905369 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 23 09:54:38.905491 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.905377 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:54:38.906142 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.906126 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:54:38.906192 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.906154 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:54:38.906192 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.906164 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:54:38.906577 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.906550 2580 policy_none.go:49] "None policy: Start" Jul 23 09:54:38.906608 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.906583 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 09:54:38.906608 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.906595 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 09:54:38.907530 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.907520 2580 policy_none.go:47] "Start" Jul 23 09:54:38.942542 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.942309 2580 manager.go:337] "Starting Device Plugin manager" Jul 23 09:54:38.942542 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:38.942541 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 09:54:38.957587 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.942572 2580 server.go:87] "Starting device plugin registration server" Jul 23 09:54:38.957587 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.942884 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 09:54:38.957587 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.942898 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 09:54:38.957587 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.943012 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 09:54:38.957587 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.943107 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 09:54:38.957587 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:38.943122 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 09:54:38.957587 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:38.943736 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 09:54:38.957587 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:38.943781 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:39.021077 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.021037 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 09:54:39.021235 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.021089 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 09:54:39.021235 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.021112 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 09:54:39.021235 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.021154 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 09:54:39.023304 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.023276 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:54:39.043710 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.043582 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:54:39.044591 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.044556 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:54:39.044719 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.044606 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:54:39.044719 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.044622 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:54:39.044719 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.044644 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.054990 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.054962 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.054990 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.054992 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-55.ec2.internal\": node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:39.070494 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.070461 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:39.121510 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.121473 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal"] Jul 23 09:54:39.121716 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.121600 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:54:39.123436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.123420 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:54:39.123512 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.123450 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:54:39.123512 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.123459 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:54:39.125802 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.125788 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:54:39.125913 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.125898 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.125953 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.125930 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:54:39.127514 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.127494 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:54:39.127598 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.127529 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:54:39.127598 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.127539 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:54:39.127598 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.127494 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:54:39.127713 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.127599 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:54:39.127713 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.127612 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:54:39.129848 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.129831 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.129925 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.129860 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:54:39.131288 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.131267 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:54:39.131388 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.131302 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:54:39.131388 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.131312 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:54:39.164497 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.164471 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-55.ec2.internal\" not found" node="ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.169017 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.168999 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-55.ec2.internal\" not found" node="ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.171094 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.171077 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:39.185459 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.185428 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/113a3405109c72a54d1ec71da895ec11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"113a3405109c72a54d1ec71da895ec11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.185547 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.185466 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/113a3405109c72a54d1ec71da895ec11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"113a3405109c72a54d1ec71da895ec11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.185547 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.185491 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d71d6e1134f838d6a77fe78231efcb8f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-55.ec2.internal\" (UID: \"d71d6e1134f838d6a77fe78231efcb8f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.272079 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.272045 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:39.286489 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.286453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d71d6e1134f838d6a77fe78231efcb8f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-55.ec2.internal\" (UID: \"d71d6e1134f838d6a77fe78231efcb8f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.286652 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.286497 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/113a3405109c72a54d1ec71da895ec11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"113a3405109c72a54d1ec71da895ec11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.286652 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.286522 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/113a3405109c72a54d1ec71da895ec11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"113a3405109c72a54d1ec71da895ec11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.286652 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.286593 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/113a3405109c72a54d1ec71da895ec11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"113a3405109c72a54d1ec71da895ec11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.286652 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.286601 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/113a3405109c72a54d1ec71da895ec11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal\" (UID: \"113a3405109c72a54d1ec71da895ec11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.286652 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.286601 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d71d6e1134f838d6a77fe78231efcb8f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-55.ec2.internal\" (UID: \"d71d6e1134f838d6a77fe78231efcb8f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.372994 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.372915 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:39.467576 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.467532 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.472396 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.472310 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" Jul 23 09:54:39.473365 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.473345 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:39.573642 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.573605 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:39.674219 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.674138 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:39.774720 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.774688 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:39.819201 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.819173 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 09:54:39.819711 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.819325 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 09:54:39.875339 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.875304 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:39.877877 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.877852 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:54:39.880877 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.880839 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 09:49:40 +0000 UTC" deadline="2028-02-15 04:12:48.629418991 +0000 UTC" Jul 23 09:54:39.880992 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:39.880890 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13722h18m8.748547503s" Jul 23 09:54:39.975810 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:39.975727 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:40.031860 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:40.031822 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113a3405109c72a54d1ec71da895ec11.slice/crio-9e55b0f39128c6cfefb1d705746c5eb2b70151f397387eac2c65f2dd872449ea WatchSource:0}: Error finding container 9e55b0f39128c6cfefb1d705746c5eb2b70151f397387eac2c65f2dd872449ea: Status 404 returned error can't find the container with id 9e55b0f39128c6cfefb1d705746c5eb2b70151f397387eac2c65f2dd872449ea Jul 23 09:54:40.032394 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:40.032373 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd71d6e1134f838d6a77fe78231efcb8f.slice/crio-4c92a04c35b34391795c5ddd1f786151b69a07822a98fe130255b29a4e69a65f WatchSource:0}: Error finding container 4c92a04c35b34391795c5ddd1f786151b69a07822a98fe130255b29a4e69a65f: Status 404 returned error can't find the container with id 4c92a04c35b34391795c5ddd1f786151b69a07822a98fe130255b29a4e69a65f Jul 23 09:54:40.037809 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:40.037791 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 09:54:40.075940 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:40.075906 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:40.078493 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:40.078471 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:54:40.176682 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:40.176635 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:40.277209 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:40.277136 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:41.630634 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:41.630596 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-55.ec2.internal\" not found" Jul 23 09:54:41.685256 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:41.685229 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:54:41.736069 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:41.736033 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" Jul 23 09:54:41.748050 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:41.748023 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 09:54:41.749252 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:41.749232 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" Jul 23 09:54:41.754781 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:41.754755 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 09:54:42.111235 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.111203 2580 apiserver.go:52] "Watching apiserver" Jul 23 09:54:42.116384 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.116353 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 09:54:42.117569 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.117533 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-dkm2f","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr","openshift-cluster-node-tuning-operator/tuned-v7kr7","openshift-image-registry/node-ca-kbzkh","openshift-multus/multus-additional-cni-plugins-h9k5p","openshift-multus/network-metrics-daemon-h95kb","kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal","openshift-dns/node-resolver-lcx56","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal","openshift-multus/multus-5dvjs","openshift-network-diagnostics/network-check-target-kd464","openshift-network-operator/iptables-alerter-bc99v","openshift-ovn-kubernetes/ovnkube-node-q4p8r","kube-system/global-pull-secret-syncer-vw75n"] Jul 23 09:54:42.120569 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.120526 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.122667 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.122619 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 09:54:42.122764 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.122693 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 09:54:42.123001 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.122975 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.123105 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.123086 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 09:54:42.123300 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.123285 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 09:54:42.123921 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.123848 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t96tt\"" Jul 23 09:54:42.124693 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.124671 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 09:54:42.125065 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.124900 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 09:54:42.125065 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.124928 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rshdf\"" Jul 23 09:54:42.125219 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.125109 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 09:54:42.125271 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.125220 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 09:54:42.125445 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.125423 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.125517 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.125456 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 09:54:42.125641 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.125628 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 09:54:42.126985 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.126967 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 09:54:42.127298 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.127278 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fgkx9\"" Jul 23 09:54:42.127487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.127469 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 09:54:42.127701 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.127685 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kbzkh" Jul 23 09:54:42.129643 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.129620 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 09:54:42.130970 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.129900 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 09:54:42.130970 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.130077 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 09:54:42.130970 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.130253 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jcsmx\"" Jul 23 09:54:42.130970 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.130503 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.132122 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.132105 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 09:54:42.132483 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.132465 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7pvrs\"" Jul 23 09:54:42.132715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.132702 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 09:54:42.132886 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.132873 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:42.132971 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.132948 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:54:42.135366 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.135346 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:42.135518 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.135427 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:54:42.136606 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.136579 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 09:54:42.137460 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.137441 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lcx56" Jul 23 09:54:42.139215 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.139067 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q7zm8\"" Jul 23 09:54:42.140921 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.139592 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 09:54:42.140921 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.140061 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 09:54:42.140921 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.140278 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.142159 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.142137 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 09:54:42.142508 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.142489 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 09:54:42.142594 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.142525 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 09:54:42.142594 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.142540 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gl2rf\"" Jul 23 09:54:42.142756 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.142741 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:54:42.144580 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.144545 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 09:54:42.144790 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.144775 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 09:54:42.144990 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.144975 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:42.145157 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.145136 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:54:42.145323 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.144980 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2hpvn\"" Jul 23 09:54:42.146358 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.146338 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.147989 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a666fcc-084b-4948-8fac-8b7824ed1b62-hosts-file\") pod \"node-resolver-lcx56\" (UID: \"4a666fcc-084b-4948-8fac-8b7824ed1b62\") " pod="openshift-dns/node-resolver-lcx56" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-sysctl-d\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-os-release\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ef627b06-7dad-475b-b4b0-2908731467e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-modprobe-d\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148153 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-kubernetes\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148183 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbpvw\" (UniqueName: \"kubernetes.io/projected/f1976a26-c857-436e-98a0-86c8a92a0818-kube-api-access-vbpvw\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148234 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-var-lib-cni-bin\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148264 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-var-lib-openvswitch\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148339 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/12d1431e-93fe-4b29-9dbc-eae156e6f45e-ovnkube-script-lib\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148362 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-sysctl-conf\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148390 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-system-cni-dir\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148419 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-var-lib-kubelet\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148447 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-run-multus-certs\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148475 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-node-log\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.150249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148500 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-hostroot\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148559 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-conf-dir\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-systemd-units\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148613 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-var-lib-kubelet\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148636 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f1976a26-c857-436e-98a0-86c8a92a0818-etc-tuned\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148674 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-socket-dir-parent\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148703 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-run-netns\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148730 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148760 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqqfd\" (UniqueName: \"kubernetes.io/projected/12d1431e-93fe-4b29-9dbc-eae156e6f45e-kube-api-access-vqqfd\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148833 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-sysconfig\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.148933 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1976a26-c857-436e-98a0-86c8a92a0818-tmp\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149010 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149062 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-cni-dir\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-run-openvswitch\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149136 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-cni-bin\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.151019 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149172 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/12d1431e-93fe-4b29-9dbc-eae156e6f45e-env-overrides\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149226 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnt94\" (UniqueName: \"kubernetes.io/projected/ef627b06-7dad-475b-b4b0-2908731467e1-kube-api-access-fnt94\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149273 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-run\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149326 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-sys\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-host\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c83bbb36-0921-4e7e-a036-c6fad9d60e43-host\") pod \"node-ca-kbzkh\" (UID: \"c83bbb36-0921-4e7e-a036-c6fad9d60e43\") " pod="openshift-image-registry/node-ca-kbzkh" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/566f324b-9a5b-4829-8ed1-69b00afb4c0d-dbus\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149439 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bc99v" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149463 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-run-systemd\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149497 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-run-ovn-kubernetes\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149553 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a666fcc-084b-4948-8fac-8b7824ed1b62-tmp-dir\") pod \"node-resolver-lcx56\" (UID: \"4a666fcc-084b-4948-8fac-8b7824ed1b62\") " pod="openshift-dns/node-resolver-lcx56" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149583 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-run-netns\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149604 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-cni-netd\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149636 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-systemd\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149676 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twm66\" (UniqueName: \"kubernetes.io/projected/ff25076a-6283-4ece-ac45-32b87379bbdd-kube-api-access-twm66\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149707 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-cnibin\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149762 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-os-release\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.152056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149792 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-var-lib-cni-multus\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149826 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-etc-kubernetes\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-log-socket\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149886 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-daemon-config\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149917 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-slash\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149947 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.149971 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-kubelet\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.150009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/12d1431e-93fe-4b29-9dbc-eae156e6f45e-ovnkube-config\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.150029 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-run-k8s-cni-cncf-io\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.150116 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n76sx\" (UniqueName: \"kubernetes.io/projected/f1a41df2-3624-4d86-9851-cb67975b09ac-kube-api-access-n76sx\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.150148 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-system-cni-dir\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.150181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef627b06-7dad-475b-b4b0-2908731467e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.150251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5llc\" (UniqueName: \"kubernetes.io/projected/c83bbb36-0921-4e7e-a036-c6fad9d60e43-kube-api-access-d5llc\") pod \"node-ca-kbzkh\" (UID: \"c83bbb36-0921-4e7e-a036-c6fad9d60e43\") " pod="openshift-image-registry/node-ca-kbzkh" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.150293 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/566f324b-9a5b-4829-8ed1-69b00afb4c0d-kubelet-config\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.150434 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-run-ovn\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.150906 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmlmn\" (UniqueName: \"kubernetes.io/projected/4a666fcc-084b-4948-8fac-8b7824ed1b62-kube-api-access-nmlmn\") pod \"node-resolver-lcx56\" (UID: \"4a666fcc-084b-4948-8fac-8b7824ed1b62\") " pod="openshift-dns/node-resolver-lcx56" Jul 23 09:54:42.152861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.151117 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-lib-modules\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.153426 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.151153 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c83bbb36-0921-4e7e-a036-c6fad9d60e43-serviceca\") pod \"node-ca-kbzkh\" (UID: \"c83bbb36-0921-4e7e-a036-c6fad9d60e43\") " pod="openshift-image-registry/node-ca-kbzkh" Jul 23 09:54:42.153426 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.151224 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1a41df2-3624-4d86-9851-cb67975b09ac-cni-binary-copy\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.153426 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.151260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-etc-openvswitch\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.153426 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.151304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/12d1431e-93fe-4b29-9dbc-eae156e6f45e-ovn-node-metrics-cert\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.153426 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.151332 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-cnibin\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.153426 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.151365 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ef627b06-7dad-475b-b4b0-2908731467e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.153426 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.152888 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 09:54:42.153426 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.153074 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 09:54:42.153426 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.153237 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 09:54:42.153426 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.153378 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r68nk\"" Jul 23 09:54:42.169254 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.169181 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2kcfj" Jul 23 09:54:42.174255 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.174229 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2kcfj" Jul 23 09:54:42.237991 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.237953 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 09:54:42.252500 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252456 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-lib-modules\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.252500 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252488 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c83bbb36-0921-4e7e-a036-c6fad9d60e43-serviceca\") pod \"node-ca-kbzkh\" (UID: \"c83bbb36-0921-4e7e-a036-c6fad9d60e43\") " pod="openshift-image-registry/node-ca-kbzkh" Jul 23 09:54:42.252725 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1a41df2-3624-4d86-9851-cb67975b09ac-cni-binary-copy\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.252725 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-socket-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.252725 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-registration-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.252725 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-device-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.252725 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252700 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-lib-modules\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.252725 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252714 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-etc-selinux\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.253000 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252739 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-etc-openvswitch\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.253000 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252765 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/12d1431e-93fe-4b29-9dbc-eae156e6f45e-ovn-node-metrics-cert\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.253000 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252804 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-cnibin\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.253000 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252829 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ef627b06-7dad-475b-b4b0-2908731467e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.253000 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a666fcc-084b-4948-8fac-8b7824ed1b62-hosts-file\") pod \"node-resolver-lcx56\" (UID: \"4a666fcc-084b-4948-8fac-8b7824ed1b62\") " pod="openshift-dns/node-resolver-lcx56" Jul 23 09:54:42.253000 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252896 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5baa7946-10bb-42ca-bd72-bd64914f2d91-agent-certs\") pod \"konnectivity-agent-dkm2f\" (UID: \"5baa7946-10bb-42ca-bd72-bd64914f2d91\") " pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:54:42.253000 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252918 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-sysctl-d\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.253000 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-os-release\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.253000 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ef627b06-7dad-475b-b4b0-2908731467e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.253000 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.252987 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-modprobe-d\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-kubernetes\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253051 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbpvw\" (UniqueName: \"kubernetes.io/projected/f1976a26-c857-436e-98a0-86c8a92a0818-kube-api-access-vbpvw\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253077 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-var-lib-cni-bin\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253100 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-var-lib-openvswitch\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/12d1431e-93fe-4b29-9dbc-eae156e6f45e-ovnkube-script-lib\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253152 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/000c5b44-02b0-4f5c-a70a-cce786383305-iptables-alerter-script\") pod \"iptables-alerter-bc99v\" (UID: \"000c5b44-02b0-4f5c-a70a-cce786383305\") " pod="openshift-network-operator/iptables-alerter-bc99v" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253179 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-sys-fs\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253221 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-sysctl-conf\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253263 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c83bbb36-0921-4e7e-a036-c6fad9d60e43-serviceca\") pod \"node-ca-kbzkh\" (UID: \"c83bbb36-0921-4e7e-a036-c6fad9d60e43\") " pod="openshift-image-registry/node-ca-kbzkh" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253341 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a666fcc-084b-4948-8fac-8b7824ed1b62-hosts-file\") pod \"node-resolver-lcx56\" (UID: \"4a666fcc-084b-4948-8fac-8b7824ed1b62\") " pod="openshift-dns/node-resolver-lcx56" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253345 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1a41df2-3624-4d86-9851-cb67975b09ac-cni-binary-copy\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253364 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-system-cni-dir\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253260 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-system-cni-dir\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253414 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-var-lib-cni-bin\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253416 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-var-lib-kubelet\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-run-multus-certs\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253459 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-var-lib-kubelet\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.253623 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-node-log\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253490 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-sysctl-d\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253511 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-etc-openvswitch\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253514 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg6qd\" (UniqueName: \"kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd\") pod \"network-check-target-kd464\" (UID: \"c72345fc-45c2-4b0a-9db9-593ae230265e\") " pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253530 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ef627b06-7dad-475b-b4b0-2908731467e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253538 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-var-lib-openvswitch\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253549 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253600 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-cnibin\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253603 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-hostroot\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253636 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-node-log\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-conf-dir\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253759 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-sysctl-conf\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253792 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-hostroot\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253844 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-conf-dir\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-systemd-units\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253922 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/000c5b44-02b0-4f5c-a70a-cce786383305-host-slash\") pod \"iptables-alerter-bc99v\" (UID: \"000c5b44-02b0-4f5c-a70a-cce786383305\") " pod="openshift-network-operator/iptables-alerter-bc99v" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253931 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-os-release\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253959 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-var-lib-kubelet\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.254462 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253974 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.253991 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-kubernetes\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254004 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f1976a26-c857-436e-98a0-86c8a92a0818-etc-tuned\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-socket-dir-parent\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254040 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-run-multus-certs\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254072 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ef627b06-7dad-475b-b4b0-2908731467e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-run-netns\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254113 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqqfd\" (UniqueName: \"kubernetes.io/projected/12d1431e-93fe-4b29-9dbc-eae156e6f45e-kube-api-access-vqqfd\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254183 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-modprobe-d\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-sysconfig\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254241 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/12d1431e-93fe-4b29-9dbc-eae156e6f45e-ovnkube-script-lib\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254253 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-systemd-units\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1976a26-c857-436e-98a0-86c8a92a0818-tmp\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254301 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-var-lib-kubelet\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.254325 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-cni-dir\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.255487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254337 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-run-netns\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.254417 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret podName:566f324b-9a5b-4829-8ed1-69b00afb4c0d nodeName:}" failed. No retries permitted until 2026-07-23 09:54:42.754394983 +0000 UTC m=+3.001351241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret") pod "global-pull-secret-syncer-vw75n" (UID: "566f324b-9a5b-4829-8ed1-69b00afb4c0d") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.254421 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254458 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.254468 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs podName:ff25076a-6283-4ece-ac45-32b87379bbdd nodeName:}" failed. No retries permitted until 2026-07-23 09:54:42.754452779 +0000 UTC m=+3.001409029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs") pod "network-metrics-daemon-h95kb" (UID: "ff25076a-6283-4ece-ac45-32b87379bbdd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254488 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-run-openvswitch\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254578 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-cni-dir\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-cni-bin\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254651 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/12d1431e-93fe-4b29-9dbc-eae156e6f45e-env-overrides\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254676 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnt94\" (UniqueName: \"kubernetes.io/projected/ef627b06-7dad-475b-b4b0-2908731467e1-kube-api-access-fnt94\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254695 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-run-openvswitch\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254748 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-sysconfig\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254780 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-run\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254807 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-sys\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-host\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254867 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c83bbb36-0921-4e7e-a036-c6fad9d60e43-host\") pod \"node-ca-kbzkh\" (UID: \"c83bbb36-0921-4e7e-a036-c6fad9d60e43\") " pod="openshift-image-registry/node-ca-kbzkh" Jul 23 09:54:42.256504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/566f324b-9a5b-4829-8ed1-69b00afb4c0d-dbus\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-run-systemd\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.254978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-run-ovn-kubernetes\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255008 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255038 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a666fcc-084b-4948-8fac-8b7824ed1b62-tmp-dir\") pod \"node-resolver-lcx56\" (UID: \"4a666fcc-084b-4948-8fac-8b7824ed1b62\") " pod="openshift-dns/node-resolver-lcx56" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-run-netns\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-cni-netd\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-systemd\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twm66\" (UniqueName: \"kubernetes.io/projected/ff25076a-6283-4ece-ac45-32b87379bbdd-kube-api-access-twm66\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255255 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-cnibin\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255281 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-os-release\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-var-lib-cni-multus\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255332 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-etc-kubernetes\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-log-socket\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-daemon-config\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-slash\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255435 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.257456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255464 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d4dj\" (UniqueName: \"kubernetes.io/projected/000c5b44-02b0-4f5c-a70a-cce786383305-kube-api-access-7d4dj\") pod \"iptables-alerter-bc99v\" (UID: \"000c5b44-02b0-4f5c-a70a-cce786383305\") " pod="openshift-network-operator/iptables-alerter-bc99v" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255493 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-kubelet\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/12d1431e-93fe-4b29-9dbc-eae156e6f45e-ovnkube-config\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255541 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-run-k8s-cni-cncf-io\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255582 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n76sx\" (UniqueName: \"kubernetes.io/projected/f1a41df2-3624-4d86-9851-cb67975b09ac-kube-api-access-n76sx\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255614 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-system-cni-dir\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255642 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef627b06-7dad-475b-b4b0-2908731467e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfn6c\" (UniqueName: \"kubernetes.io/projected/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-kube-api-access-gfn6c\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5llc\" (UniqueName: \"kubernetes.io/projected/c83bbb36-0921-4e7e-a036-c6fad9d60e43-kube-api-access-d5llc\") pod \"node-ca-kbzkh\" (UID: \"c83bbb36-0921-4e7e-a036-c6fad9d60e43\") " pod="openshift-image-registry/node-ca-kbzkh" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.255725 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/566f324b-9a5b-4829-8ed1-69b00afb4c0d-kubelet-config\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.256372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-run-ovn\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.256403 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5baa7946-10bb-42ca-bd72-bd64914f2d91-konnectivity-ca\") pod \"konnectivity-agent-dkm2f\" (UID: \"5baa7946-10bb-42ca-bd72-bd64914f2d91\") " pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.256432 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmlmn\" (UniqueName: \"kubernetes.io/projected/4a666fcc-084b-4948-8fac-8b7824ed1b62-kube-api-access-nmlmn\") pod \"node-resolver-lcx56\" (UID: \"4a666fcc-084b-4948-8fac-8b7824ed1b62\") " pod="openshift-dns/node-resolver-lcx56" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.256934 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/566f324b-9a5b-4829-8ed1-69b00afb4c0d-dbus\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.257161 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-socket-dir-parent\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.257223 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-cni-bin\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.257629 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/12d1431e-93fe-4b29-9dbc-eae156e6f45e-env-overrides\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.258297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.257687 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-run\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.257729 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-sys\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.257767 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-host\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.257801 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c83bbb36-0921-4e7e-a036-c6fad9d60e43-host\") pod \"node-ca-kbzkh\" (UID: \"c83bbb36-0921-4e7e-a036-c6fad9d60e43\") " pod="openshift-image-registry/node-ca-kbzkh" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.257831 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-run-systemd\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.257861 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-run-ovn-kubernetes\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258003 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/12d1431e-93fe-4b29-9dbc-eae156e6f45e-ovn-node-metrics-cert\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258090 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-slash\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258121 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a666fcc-084b-4948-8fac-8b7824ed1b62-tmp-dir\") pod \"node-resolver-lcx56\" (UID: \"4a666fcc-084b-4948-8fac-8b7824ed1b62\") " pod="openshift-dns/node-resolver-lcx56" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258204 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-run-netns\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258243 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-cni-netd\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258288 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f1976a26-c857-436e-98a0-86c8a92a0818-etc-systemd\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258330 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-cnibin\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258376 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-os-release\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258408 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-var-lib-cni-multus\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258444 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-etc-kubernetes\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258454 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f1976a26-c857-436e-98a0-86c8a92a0818-etc-tuned\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258479 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/566f324b-9a5b-4829-8ed1-69b00afb4c0d-kubelet-config\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:42.259116 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258501 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f1a41df2-3624-4d86-9851-cb67975b09ac-multus-daemon-config\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.259917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258534 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-run-ovn\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.259917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258560 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f1a41df2-3624-4d86-9851-cb67975b09ac-host-run-k8s-cni-cncf-io\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.259917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258582 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-host-kubelet\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.259917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258800 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-system-cni-dir\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.259917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258912 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef627b06-7dad-475b-b4b0-2908731467e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.259917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258937 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef627b06-7dad-475b-b4b0-2908731467e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.259917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.258974 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/12d1431e-93fe-4b29-9dbc-eae156e6f45e-ovnkube-config\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.259917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.259162 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/12d1431e-93fe-4b29-9dbc-eae156e6f45e-log-socket\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.259917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.259661 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1976a26-c857-436e-98a0-86c8a92a0818-tmp\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.264157 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.264123 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbpvw\" (UniqueName: \"kubernetes.io/projected/f1976a26-c857-436e-98a0-86c8a92a0818-kube-api-access-vbpvw\") pod \"tuned-v7kr7\" (UID: \"f1976a26-c857-436e-98a0-86c8a92a0818\") " pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.264614 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.264591 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twm66\" (UniqueName: \"kubernetes.io/projected/ff25076a-6283-4ece-ac45-32b87379bbdd-kube-api-access-twm66\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:42.264746 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.264725 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmlmn\" (UniqueName: \"kubernetes.io/projected/4a666fcc-084b-4948-8fac-8b7824ed1b62-kube-api-access-nmlmn\") pod \"node-resolver-lcx56\" (UID: \"4a666fcc-084b-4948-8fac-8b7824ed1b62\") " pod="openshift-dns/node-resolver-lcx56" Jul 23 09:54:42.264934 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.264907 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqqfd\" (UniqueName: \"kubernetes.io/projected/12d1431e-93fe-4b29-9dbc-eae156e6f45e-kube-api-access-vqqfd\") pod \"ovnkube-node-q4p8r\" (UID: \"12d1431e-93fe-4b29-9dbc-eae156e6f45e\") " pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.265331 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.265311 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnt94\" (UniqueName: \"kubernetes.io/projected/ef627b06-7dad-475b-b4b0-2908731467e1-kube-api-access-fnt94\") pod \"multus-additional-cni-plugins-h9k5p\" (UID: \"ef627b06-7dad-475b-b4b0-2908731467e1\") " pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.266134 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.266118 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n76sx\" (UniqueName: \"kubernetes.io/projected/f1a41df2-3624-4d86-9851-cb67975b09ac-kube-api-access-n76sx\") pod \"multus-5dvjs\" (UID: \"f1a41df2-3624-4d86-9851-cb67975b09ac\") " pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.268598 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.268576 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5llc\" (UniqueName: \"kubernetes.io/projected/c83bbb36-0921-4e7e-a036-c6fad9d60e43-kube-api-access-d5llc\") pod \"node-ca-kbzkh\" (UID: \"c83bbb36-0921-4e7e-a036-c6fad9d60e43\") " pod="openshift-image-registry/node-ca-kbzkh" Jul 23 09:54:42.280401 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.280342 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" event={"ID":"113a3405109c72a54d1ec71da895ec11","Type":"ContainerStarted","Data":"9e55b0f39128c6cfefb1d705746c5eb2b70151f397387eac2c65f2dd872449ea"} Jul 23 09:54:42.281270 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.281249 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" event={"ID":"d71d6e1134f838d6a77fe78231efcb8f","Type":"ContainerStarted","Data":"4c92a04c35b34391795c5ddd1f786151b69a07822a98fe130255b29a4e69a65f"} Jul 23 09:54:42.357782 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.357741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfn6c\" (UniqueName: \"kubernetes.io/projected/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-kube-api-access-gfn6c\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.357961 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.357800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5baa7946-10bb-42ca-bd72-bd64914f2d91-konnectivity-ca\") pod \"konnectivity-agent-dkm2f\" (UID: \"5baa7946-10bb-42ca-bd72-bd64914f2d91\") " pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:54:42.357961 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.357859 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-socket-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.357961 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.357891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-registration-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.357961 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.357914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-device-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.357961 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.357933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-etc-selinux\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.357961 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.357959 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5baa7946-10bb-42ca-bd72-bd64914f2d91-agent-certs\") pod \"konnectivity-agent-dkm2f\" (UID: \"5baa7946-10bb-42ca-bd72-bd64914f2d91\") " pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:54:42.358342 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.357990 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/000c5b44-02b0-4f5c-a70a-cce786383305-iptables-alerter-script\") pod \"iptables-alerter-bc99v\" (UID: \"000c5b44-02b0-4f5c-a70a-cce786383305\") " pod="openshift-network-operator/iptables-alerter-bc99v" Jul 23 09:54:42.358342 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358019 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-sys-fs\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.358342 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358038 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-registration-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.358342 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg6qd\" (UniqueName: \"kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd\") pod \"network-check-target-kd464\" (UID: \"c72345fc-45c2-4b0a-9db9-593ae230265e\") " pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:42.358342 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/000c5b44-02b0-4f5c-a70a-cce786383305-host-slash\") pod \"iptables-alerter-bc99v\" (UID: \"000c5b44-02b0-4f5c-a70a-cce786383305\") " pod="openshift-network-operator/iptables-alerter-bc99v" Jul 23 09:54:42.358342 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358131 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-socket-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.358342 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.358342 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d4dj\" (UniqueName: \"kubernetes.io/projected/000c5b44-02b0-4f5c-a70a-cce786383305-kube-api-access-7d4dj\") pod \"iptables-alerter-bc99v\" (UID: \"000c5b44-02b0-4f5c-a70a-cce786383305\") " pod="openshift-network-operator/iptables-alerter-bc99v" Jul 23 09:54:42.358342 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358258 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-device-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.358715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358374 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-etc-selinux\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.358715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358381 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-sys-fs\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.358715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358413 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5baa7946-10bb-42ca-bd72-bd64914f2d91-konnectivity-ca\") pod \"konnectivity-agent-dkm2f\" (UID: \"5baa7946-10bb-42ca-bd72-bd64914f2d91\") " pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:54:42.358715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358423 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.358715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358461 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/000c5b44-02b0-4f5c-a70a-cce786383305-host-slash\") pod \"iptables-alerter-bc99v\" (UID: \"000c5b44-02b0-4f5c-a70a-cce786383305\") " pod="openshift-network-operator/iptables-alerter-bc99v" Jul 23 09:54:42.358715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.358653 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/000c5b44-02b0-4f5c-a70a-cce786383305-iptables-alerter-script\") pod \"iptables-alerter-bc99v\" (UID: \"000c5b44-02b0-4f5c-a70a-cce786383305\") " pod="openshift-network-operator/iptables-alerter-bc99v" Jul 23 09:54:42.360973 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.360948 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5baa7946-10bb-42ca-bd72-bd64914f2d91-agent-certs\") pod \"konnectivity-agent-dkm2f\" (UID: \"5baa7946-10bb-42ca-bd72-bd64914f2d91\") " pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:54:42.364003 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.363950 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:54:42.364003 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.363972 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:54:42.364003 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.363984 2580 projected.go:196] Error preparing data for projected volume kube-api-access-sg6qd for pod openshift-network-diagnostics/network-check-target-kd464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:42.364223 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.364060 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd podName:c72345fc-45c2-4b0a-9db9-593ae230265e nodeName:}" failed. No retries permitted until 2026-07-23 09:54:42.864044501 +0000 UTC m=+3.111000765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sg6qd" (UniqueName: "kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd") pod "network-check-target-kd464" (UID: "c72345fc-45c2-4b0a-9db9-593ae230265e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:42.366513 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.366492 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d4dj\" (UniqueName: \"kubernetes.io/projected/000c5b44-02b0-4f5c-a70a-cce786383305-kube-api-access-7d4dj\") pod \"iptables-alerter-bc99v\" (UID: \"000c5b44-02b0-4f5c-a70a-cce786383305\") " pod="openshift-network-operator/iptables-alerter-bc99v" Jul 23 09:54:42.366626 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.366605 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfn6c\" (UniqueName: \"kubernetes.io/projected/9ec61f70-2f55-4d9e-aa93-e5110a45bb33-kube-api-access-gfn6c\") pod \"aws-ebs-csi-driver-node-bz9rr\" (UID: \"9ec61f70-2f55-4d9e-aa93-e5110a45bb33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.379526 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.379504 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:54:42.434247 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.434208 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5dvjs" Jul 23 09:54:42.443935 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.443897 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:54:42.455104 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.455076 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" Jul 23 09:54:42.461025 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.461004 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kbzkh" Jul 23 09:54:42.468727 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.468700 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" Jul 23 09:54:42.475735 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.475698 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lcx56" Jul 23 09:54:42.484515 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.484494 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" Jul 23 09:54:42.492351 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.492324 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:54:42.499135 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.499114 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bc99v" Jul 23 09:54:42.761739 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.761644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:42.761739 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.761686 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:42.761954 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.761841 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:42.761954 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.761842 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:42.761954 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.761906 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs podName:ff25076a-6283-4ece-ac45-32b87379bbdd nodeName:}" failed. No retries permitted until 2026-07-23 09:54:43.761886486 +0000 UTC m=+4.008842731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs") pod "network-metrics-daemon-h95kb" (UID: "ff25076a-6283-4ece-ac45-32b87379bbdd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:42.761954 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.761921 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret podName:566f324b-9a5b-4829-8ed1-69b00afb4c0d nodeName:}" failed. No retries permitted until 2026-07-23 09:54:43.761913792 +0000 UTC m=+4.008870036 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret") pod "global-pull-secret-syncer-vw75n" (UID: "566f324b-9a5b-4829-8ed1-69b00afb4c0d") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:42.963110 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:42.963066 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg6qd\" (UniqueName: \"kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd\") pod \"network-check-target-kd464\" (UID: \"c72345fc-45c2-4b0a-9db9-593ae230265e\") " pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:42.963323 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.963236 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:54:42.963323 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.963259 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:54:42.963323 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.963272 2580 projected.go:196] Error preparing data for projected volume kube-api-access-sg6qd for pod openshift-network-diagnostics/network-check-target-kd464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:42.963496 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:42.963330 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd podName:c72345fc-45c2-4b0a-9db9-593ae230265e nodeName:}" failed. No retries permitted until 2026-07-23 09:54:43.963314761 +0000 UTC m=+4.210271023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sg6qd" (UniqueName: "kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd") pod "network-check-target-kd464" (UID: "c72345fc-45c2-4b0a-9db9-593ae230265e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:43.178039 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:43.177996 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 09:49:42 +0000 UTC" deadline="2028-04-04 21:13:17.581391982 +0000 UTC" Jul 23 09:54:43.178039 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:43.178033 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14915h18m34.403362663s" Jul 23 09:54:43.259940 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:43.259911 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc83bbb36_0921_4e7e_a036_c6fad9d60e43.slice/crio-4edfb4f34bd932c4ffb4958944bc23953991230589bad72e373b526a70704602 WatchSource:0}: Error finding container 4edfb4f34bd932c4ffb4958944bc23953991230589bad72e373b526a70704602: Status 404 returned error can't find the container with id 4edfb4f34bd932c4ffb4958944bc23953991230589bad72e373b526a70704602 Jul 23 09:54:43.260791 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:43.260767 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d1431e_93fe_4b29_9dbc_eae156e6f45e.slice/crio-3753ed216dc66998382431b1c5d6d327329b6c83394e0bcb9de791c136ddc28b WatchSource:0}: Error finding container 3753ed216dc66998382431b1c5d6d327329b6c83394e0bcb9de791c136ddc28b: Status 404 returned error can't find the container with id 3753ed216dc66998382431b1c5d6d327329b6c83394e0bcb9de791c136ddc28b Jul 23 09:54:43.286630 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:43.286596 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" event={"ID":"f1976a26-c857-436e-98a0-86c8a92a0818","Type":"ContainerStarted","Data":"bcc36c7a36033c225d43f3bc38cff4e2693308be9586d6ac469787eafe351ef2"} Jul 23 09:54:43.287761 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:43.287729 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" event={"ID":"12d1431e-93fe-4b29-9dbc-eae156e6f45e","Type":"ContainerStarted","Data":"3753ed216dc66998382431b1c5d6d327329b6c83394e0bcb9de791c136ddc28b"} Jul 23 09:54:43.288768 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:43.288741 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kbzkh" event={"ID":"c83bbb36-0921-4e7e-a036-c6fad9d60e43","Type":"ContainerStarted","Data":"4edfb4f34bd932c4ffb4958944bc23953991230589bad72e373b526a70704602"} Jul 23 09:54:43.294788 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:43.294762 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef627b06_7dad_475b_b4b0_2908731467e1.slice/crio-8efc77801e07661616cfa94f1624b3889478f2326b56e98d1820ca08ac12443f WatchSource:0}: Error finding container 8efc77801e07661616cfa94f1624b3889478f2326b56e98d1820ca08ac12443f: Status 404 returned error can't find the container with id 8efc77801e07661616cfa94f1624b3889478f2326b56e98d1820ca08ac12443f Jul 23 09:54:43.295872 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:43.295846 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000c5b44_02b0_4f5c_a70a_cce786383305.slice/crio-a888ec1d4c1e197205f06f31c9e379eb80f919607535ac43b48f7272f0023cff WatchSource:0}: Error finding container a888ec1d4c1e197205f06f31c9e379eb80f919607535ac43b48f7272f0023cff: Status 404 returned error can't find the container with id a888ec1d4c1e197205f06f31c9e379eb80f919607535ac43b48f7272f0023cff Jul 23 09:54:43.296802 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:43.296776 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec61f70_2f55_4d9e_aa93_e5110a45bb33.slice/crio-d8a4cb5fccbf1695fa01d97c068a0c984970ebaec9397f6daf09551b8abf2973 WatchSource:0}: Error finding container d8a4cb5fccbf1695fa01d97c068a0c984970ebaec9397f6daf09551b8abf2973: Status 404 returned error can't find the container with id d8a4cb5fccbf1695fa01d97c068a0c984970ebaec9397f6daf09551b8abf2973 Jul 23 09:54:43.297311 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:43.297293 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a666fcc_084b_4948_8fac_8b7824ed1b62.slice/crio-ad80e098a0e1daf2e5e7fe24c1b9eb18aa3cf6182bcf6cda362a27d178b44b17 WatchSource:0}: Error finding container ad80e098a0e1daf2e5e7fe24c1b9eb18aa3cf6182bcf6cda362a27d178b44b17: Status 404 returned error can't find the container with id ad80e098a0e1daf2e5e7fe24c1b9eb18aa3cf6182bcf6cda362a27d178b44b17 Jul 23 09:54:43.299704 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:43.299680 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5baa7946_10bb_42ca_bd72_bd64914f2d91.slice/crio-61ca8edd06c308f99d9e7f6a78b1c8623f59bce737c46fc65ba20c6486c3f660 WatchSource:0}: Error finding container 61ca8edd06c308f99d9e7f6a78b1c8623f59bce737c46fc65ba20c6486c3f660: Status 404 returned error can't find the container with id 61ca8edd06c308f99d9e7f6a78b1c8623f59bce737c46fc65ba20c6486c3f660 Jul 23 09:54:43.300343 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:54:43.300087 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a41df2_3624_4d86_9851_cb67975b09ac.slice/crio-074f7081f71dbcbd90dde3f7151afc5cb64cf9601a23cb38994ae5c409f3a83b WatchSource:0}: Error finding container 074f7081f71dbcbd90dde3f7151afc5cb64cf9601a23cb38994ae5c409f3a83b: Status 404 returned error can't find the container with id 074f7081f71dbcbd90dde3f7151afc5cb64cf9601a23cb38994ae5c409f3a83b Jul 23 09:54:43.773129 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:43.772607 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:43.773129 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:43.772860 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:43.773129 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:43.772906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:43.773129 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:43.772936 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret podName:566f324b-9a5b-4829-8ed1-69b00afb4c0d nodeName:}" failed. No retries permitted until 2026-07-23 09:54:45.772915397 +0000 UTC m=+6.019871647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret") pod "global-pull-secret-syncer-vw75n" (UID: "566f324b-9a5b-4829-8ed1-69b00afb4c0d") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:43.773129 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:43.772994 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:43.773129 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:43.773048 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs podName:ff25076a-6283-4ece-ac45-32b87379bbdd nodeName:}" failed. No retries permitted until 2026-07-23 09:54:45.773033064 +0000 UTC m=+6.019989324 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs") pod "network-metrics-daemon-h95kb" (UID: "ff25076a-6283-4ece-ac45-32b87379bbdd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:43.974733 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:43.974677 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg6qd\" (UniqueName: \"kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd\") pod \"network-check-target-kd464\" (UID: \"c72345fc-45c2-4b0a-9db9-593ae230265e\") " pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:43.974917 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:43.974893 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:54:43.974917 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:43.974911 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:54:43.975023 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:43.974925 2580 projected.go:196] Error preparing data for projected volume kube-api-access-sg6qd for pod openshift-network-diagnostics/network-check-target-kd464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:43.975023 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:43.974983 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd podName:c72345fc-45c2-4b0a-9db9-593ae230265e nodeName:}" failed. No retries permitted until 2026-07-23 09:54:45.974965804 +0000 UTC m=+6.221922053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sg6qd" (UniqueName: "kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd") pod "network-check-target-kd464" (UID: "c72345fc-45c2-4b0a-9db9-593ae230265e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:44.178816 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.178697 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 09:49:42 +0000 UTC" deadline="2028-01-09 14:57:03.110356084 +0000 UTC" Jul 23 09:54:44.178816 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.178767 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12845h2m18.931593304s" Jul 23 09:54:44.283826 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.282873 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:44.283826 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:44.282988 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:54:44.283826 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.283435 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:44.283826 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:44.283535 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:54:44.283826 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.283601 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:44.283826 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:44.283672 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:54:44.296381 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.296347 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" event={"ID":"ef627b06-7dad-475b-b4b0-2908731467e1","Type":"ContainerStarted","Data":"8efc77801e07661616cfa94f1624b3889478f2326b56e98d1820ca08ac12443f"} Jul 23 09:54:44.299009 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.298974 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5dvjs" event={"ID":"f1a41df2-3624-4d86-9851-cb67975b09ac","Type":"ContainerStarted","Data":"074f7081f71dbcbd90dde3f7151afc5cb64cf9601a23cb38994ae5c409f3a83b"} Jul 23 09:54:44.305104 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.305065 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dkm2f" event={"ID":"5baa7946-10bb-42ca-bd72-bd64914f2d91","Type":"ContainerStarted","Data":"61ca8edd06c308f99d9e7f6a78b1c8623f59bce737c46fc65ba20c6486c3f660"} Jul 23 09:54:44.316395 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.316355 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" event={"ID":"9ec61f70-2f55-4d9e-aa93-e5110a45bb33","Type":"ContainerStarted","Data":"d8a4cb5fccbf1695fa01d97c068a0c984970ebaec9397f6daf09551b8abf2973"} Jul 23 09:54:44.327420 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.327231 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" event={"ID":"113a3405109c72a54d1ec71da895ec11","Type":"ContainerDied","Data":"72099c241b588307de2f9de258113e3af300dc044a29e0e40e10959657ff04af"} Jul 23 09:54:44.327420 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.326179 2580 generic.go:356] "Generic (PLEG): container finished" podID="113a3405109c72a54d1ec71da895ec11" containerID="72099c241b588307de2f9de258113e3af300dc044a29e0e40e10959657ff04af" exitCode=0 Jul 23 09:54:44.348955 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.348914 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" event={"ID":"d71d6e1134f838d6a77fe78231efcb8f","Type":"ContainerStarted","Data":"22db5353c6b17a48dd7f8050fdea3fa7a4d12e7c19d0935c941bee8eb71a3c5c"} Jul 23 09:54:44.357883 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.357840 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lcx56" event={"ID":"4a666fcc-084b-4948-8fac-8b7824ed1b62","Type":"ContainerStarted","Data":"ad80e098a0e1daf2e5e7fe24c1b9eb18aa3cf6182bcf6cda362a27d178b44b17"} Jul 23 09:54:44.381166 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:44.381125 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bc99v" event={"ID":"000c5b44-02b0-4f5c-a70a-cce786383305","Type":"ContainerStarted","Data":"a888ec1d4c1e197205f06f31c9e379eb80f919607535ac43b48f7272f0023cff"} Jul 23 09:54:45.396052 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:45.395925 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" event={"ID":"113a3405109c72a54d1ec71da895ec11","Type":"ContainerStarted","Data":"f898d1993ae24a6156dc55f134cb92c7460ad1237df0770dc937c8f99bcfd866"} Jul 23 09:54:45.411723 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:45.411250 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-55.ec2.internal" podStartSLOduration=4.411227174 podStartE2EDuration="4.411227174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:54:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:54:44.361859074 +0000 UTC m=+4.608815343" watchObservedRunningTime="2026-07-23 09:54:45.411227174 +0000 UTC m=+5.658183443" Jul 23 09:54:45.790252 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:45.790149 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:45.790415 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:45.790299 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:45.790485 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:45.790428 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:45.790536 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:45.790490 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret podName:566f324b-9a5b-4829-8ed1-69b00afb4c0d nodeName:}" failed. No retries permitted until 2026-07-23 09:54:49.790470794 +0000 UTC m=+10.037427041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret") pod "global-pull-secret-syncer-vw75n" (UID: "566f324b-9a5b-4829-8ed1-69b00afb4c0d") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:45.790966 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:45.790878 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:45.790966 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:45.790928 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs podName:ff25076a-6283-4ece-ac45-32b87379bbdd nodeName:}" failed. No retries permitted until 2026-07-23 09:54:49.79091166 +0000 UTC m=+10.037867905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs") pod "network-metrics-daemon-h95kb" (UID: "ff25076a-6283-4ece-ac45-32b87379bbdd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:45.992471 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:45.992434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg6qd\" (UniqueName: \"kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd\") pod \"network-check-target-kd464\" (UID: \"c72345fc-45c2-4b0a-9db9-593ae230265e\") " pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:45.992673 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:45.992605 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:54:45.992673 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:45.992627 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:54:45.992673 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:45.992639 2580 projected.go:196] Error preparing data for projected volume kube-api-access-sg6qd for pod openshift-network-diagnostics/network-check-target-kd464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:45.992829 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:45.992698 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd podName:c72345fc-45c2-4b0a-9db9-593ae230265e nodeName:}" failed. No retries permitted until 2026-07-23 09:54:49.992680282 +0000 UTC m=+10.239636546 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sg6qd" (UniqueName: "kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd") pod "network-check-target-kd464" (UID: "c72345fc-45c2-4b0a-9db9-593ae230265e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:46.274479 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:46.274384 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:46.274649 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:46.274527 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:54:46.274988 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:46.274950 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:46.275118 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:46.275062 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:54:46.275218 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:46.275121 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:46.275279 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:46.275225 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:54:48.277760 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:48.277725 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:48.278267 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:48.277845 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:54:48.278357 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:48.278279 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:48.278415 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:48.278387 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:54:48.278467 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:48.278453 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:48.278550 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:48.278525 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:54:49.822228 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:49.822170 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:49.822228 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:49.822244 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:49.822702 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:49.822372 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:49.822702 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:49.822434 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs podName:ff25076a-6283-4ece-ac45-32b87379bbdd nodeName:}" failed. No retries permitted until 2026-07-23 09:54:57.822415506 +0000 UTC m=+18.069371755 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs") pod "network-metrics-daemon-h95kb" (UID: "ff25076a-6283-4ece-ac45-32b87379bbdd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:49.822803 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:49.822788 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:49.822841 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:49.822831 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret podName:566f324b-9a5b-4829-8ed1-69b00afb4c0d nodeName:}" failed. No retries permitted until 2026-07-23 09:54:57.822820214 +0000 UTC m=+18.069776459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret") pod "global-pull-secret-syncer-vw75n" (UID: "566f324b-9a5b-4829-8ed1-69b00afb4c0d") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:50.024641 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:50.024610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg6qd\" (UniqueName: \"kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd\") pod \"network-check-target-kd464\" (UID: \"c72345fc-45c2-4b0a-9db9-593ae230265e\") " pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:50.024811 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:50.024773 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:54:50.024811 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:50.024789 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:54:50.024811 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:50.024798 2580 projected.go:196] Error preparing data for projected volume kube-api-access-sg6qd for pod openshift-network-diagnostics/network-check-target-kd464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:50.024928 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:50.024842 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd podName:c72345fc-45c2-4b0a-9db9-593ae230265e nodeName:}" failed. No retries permitted until 2026-07-23 09:54:58.024827738 +0000 UTC m=+18.271783982 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sg6qd" (UniqueName: "kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd") pod "network-check-target-kd464" (UID: "c72345fc-45c2-4b0a-9db9-593ae230265e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:50.275261 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:50.275155 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:50.275261 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:50.275202 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:50.275492 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:50.275302 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:54:50.275492 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:50.275323 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:50.275492 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:50.275417 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:54:50.275643 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:50.275492 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:54:52.275957 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:52.275318 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:52.275957 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:52.275450 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:54:52.275957 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:52.275821 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:52.275957 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:52.275908 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:54:52.276744 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:52.276610 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:52.276744 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:52.276705 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:54:54.274508 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:54.274419 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:54.274508 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:54.274446 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:54.274508 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:54.274419 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:54.275068 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:54.274574 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:54:54.275068 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:54.274625 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:54:54.275068 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:54.274664 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:54:56.274850 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:56.274783 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:56.275284 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:56.274788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:56.275284 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:56.274921 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:54:56.275284 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:56.274949 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:56.275284 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:56.275043 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:54:56.275284 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:56.275135 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:54:57.887269 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:57.887233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:57.887269 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:57.887275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:57.887823 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:57.887403 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:57.887823 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:57.887409 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:57.887823 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:57.887466 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs podName:ff25076a-6283-4ece-ac45-32b87379bbdd nodeName:}" failed. No retries permitted until 2026-07-23 09:55:13.887450975 +0000 UTC m=+34.134407219 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs") pod "network-metrics-daemon-h95kb" (UID: "ff25076a-6283-4ece-ac45-32b87379bbdd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:54:57.887823 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:57.887480 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret podName:566f324b-9a5b-4829-8ed1-69b00afb4c0d nodeName:}" failed. No retries permitted until 2026-07-23 09:55:13.887473686 +0000 UTC m=+34.134429930 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret") pod "global-pull-secret-syncer-vw75n" (UID: "566f324b-9a5b-4829-8ed1-69b00afb4c0d") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:54:58.088613 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:58.088570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg6qd\" (UniqueName: \"kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd\") pod \"network-check-target-kd464\" (UID: \"c72345fc-45c2-4b0a-9db9-593ae230265e\") " pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:58.088807 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:58.088768 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:54:58.088807 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:58.088792 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:54:58.088807 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:58.088804 2580 projected.go:196] Error preparing data for projected volume kube-api-access-sg6qd for pod openshift-network-diagnostics/network-check-target-kd464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:58.088951 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:58.088870 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd podName:c72345fc-45c2-4b0a-9db9-593ae230265e nodeName:}" failed. No retries permitted until 2026-07-23 09:55:14.088851566 +0000 UTC m=+34.335807831 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sg6qd" (UniqueName: "kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd") pod "network-check-target-kd464" (UID: "c72345fc-45c2-4b0a-9db9-593ae230265e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:54:58.274828 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:58.274795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:54:58.275002 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:58.274795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:54:58.275002 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:58.274907 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:54:58.275002 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:54:58.274794 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:54:58.275140 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:58.275012 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:54:58.275140 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:54:58.275081 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:55:00.275630 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:00.275591 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:00.276034 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:00.275754 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:55:00.276208 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:00.276167 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:00.276350 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:00.276283 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:55:00.276350 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:00.276327 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:00.276468 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:00.276366 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:55:02.275502 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.275322 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:02.275502 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.275321 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:02.275502 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:02.275463 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:55:02.275974 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:02.275533 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:55:02.275974 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.275565 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:02.275974 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:02.275637 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:55:02.432485 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.432294 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" event={"ID":"12d1431e-93fe-4b29-9dbc-eae156e6f45e","Type":"ContainerStarted","Data":"82f5493c3b5b0e5bef1e258a132baf8654be8c2aed5c707490e20a352dc36905"} Jul 23 09:55:02.436380 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.436346 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kbzkh" event={"ID":"c83bbb36-0921-4e7e-a036-c6fad9d60e43","Type":"ContainerStarted","Data":"c82ae0f185a355ba280a3b46946de8712881eb8de3a008d86bc3d5a5efd79cc9"} Jul 23 09:55:02.437895 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.437872 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lcx56" event={"ID":"4a666fcc-084b-4948-8fac-8b7824ed1b62","Type":"ContainerStarted","Data":"406a59c46dc50f1d7df994e8a870da71657c9640abb2711d3f1d9d0bed74cc4e"} Jul 23 09:55:02.439851 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.439823 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" event={"ID":"ef627b06-7dad-475b-b4b0-2908731467e1","Type":"ContainerStarted","Data":"f50bb22fd5d91da5af824ebd63a781df8a640918ee53d537e3ee548728bacbb0"} Jul 23 09:55:02.441288 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.441253 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" event={"ID":"f1976a26-c857-436e-98a0-86c8a92a0818","Type":"ContainerStarted","Data":"1bcc6b14e0f8060e29b726199c977d1eb19bd7444341707ccce8e3f1fad2d50b"} Jul 23 09:55:02.443029 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.442997 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5dvjs" event={"ID":"f1a41df2-3624-4d86-9851-cb67975b09ac","Type":"ContainerStarted","Data":"77c00978b6c53408561e56b39803922ca7e4c3f53fe222e7a7320b3e9ff8b3fd"} Jul 23 09:55:02.444513 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.444484 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dkm2f" event={"ID":"5baa7946-10bb-42ca-bd72-bd64914f2d91","Type":"ContainerStarted","Data":"451ad4a5189841326c3a0f168e0d1a1089415e7df2cbc419c09fb5057f9aab64"} Jul 23 09:55:02.445916 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.445893 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" event={"ID":"9ec61f70-2f55-4d9e-aa93-e5110a45bb33","Type":"ContainerStarted","Data":"f2b49e996350471d221fff87fe511b42a2bd2b015dd77d6237073c4dba4c2af1"} Jul 23 09:55:02.450911 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.450876 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-55.ec2.internal" podStartSLOduration=21.450864508 podStartE2EDuration="21.450864508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:54:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:54:45.411635895 +0000 UTC m=+5.658592162" watchObservedRunningTime="2026-07-23 09:55:02.450864508 +0000 UTC m=+22.697820774" Jul 23 09:55:02.451365 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.451342 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kbzkh" podStartSLOduration=3.584894717 podStartE2EDuration="22.451334741s" totalImagesPullingTime="18.866440024s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:02.450792127 +0000 UTC m=+22.697748390" watchObservedRunningTime="2026-07-23 09:55:02.451334741 +0000 UTC m=+22.698291007" Jul 23 09:55:02.471265 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.471220 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lcx56" podStartSLOduration=3.641794732 podStartE2EDuration="22.47118387s" totalImagesPullingTime="18.829389138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:02.470901727 +0000 UTC m=+22.717857992" watchObservedRunningTime="2026-07-23 09:55:02.47118387 +0000 UTC m=+22.718140135" Jul 23 09:55:02.484130 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.484069 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dkm2f" podStartSLOduration=7.538814799 podStartE2EDuration="22.484052555s" totalImagesPullingTime="14.945237756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:02.483430042 +0000 UTC m=+22.730386307" watchObservedRunningTime="2026-07-23 09:55:02.484052555 +0000 UTC m=+22.731008821" Jul 23 09:55:02.516615 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.516570 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5dvjs" podStartSLOduration=3.61154644 podStartE2EDuration="22.516554346s" totalImagesPullingTime="18.905007906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:02.516308236 +0000 UTC m=+22.763264504" watchObservedRunningTime="2026-07-23 09:55:02.516554346 +0000 UTC m=+22.763510612" Jul 23 09:55:02.532571 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:02.532522 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v7kr7" podStartSLOduration=3.63601428 podStartE2EDuration="22.532506675s" totalImagesPullingTime="18.896492395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:02.531931028 +0000 UTC m=+22.778887295" watchObservedRunningTime="2026-07-23 09:55:02.532506675 +0000 UTC m=+22.779463005" Jul 23 09:55:03.451809 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:03.451592 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 09:55:03.452620 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:03.452099 2580 generic.go:356] "Generic (PLEG): container finished" podID="12d1431e-93fe-4b29-9dbc-eae156e6f45e" containerID="2a552ee7d4159d9c9ccb129860be8d48c8cb03f31109009101f66850ae266c66" exitCode=1 Jul 23 09:55:03.452620 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:03.452216 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" event={"ID":"12d1431e-93fe-4b29-9dbc-eae156e6f45e","Type":"ContainerStarted","Data":"f3bd396989f973ba2bea919fcf28ae7220339cdb6ac96895c9e470c7d40e764e"} Jul 23 09:55:03.452620 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:03.452426 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" event={"ID":"12d1431e-93fe-4b29-9dbc-eae156e6f45e","Type":"ContainerStarted","Data":"1cab8e69b807d0c4582c257ba8b504afd0a0b5cc0bf52b0b17caaeb2316bf210"} Jul 23 09:55:03.452620 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:03.452485 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" event={"ID":"12d1431e-93fe-4b29-9dbc-eae156e6f45e","Type":"ContainerStarted","Data":"c10b85e78a5879d672af0efb9402380d7489b029abb9b070490cecbb9e023917"} Jul 23 09:55:03.452620 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:03.452532 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" event={"ID":"12d1431e-93fe-4b29-9dbc-eae156e6f45e","Type":"ContainerStarted","Data":"92777dd09e0ee552d8bfb27c1324961e62553c6ffaea70e8beb123156b594119"} Jul 23 09:55:03.452620 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:03.452575 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" event={"ID":"12d1431e-93fe-4b29-9dbc-eae156e6f45e","Type":"ContainerDied","Data":"2a552ee7d4159d9c9ccb129860be8d48c8cb03f31109009101f66850ae266c66"} Jul 23 09:55:03.453844 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:03.453817 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef627b06-7dad-475b-b4b0-2908731467e1" containerID="f50bb22fd5d91da5af824ebd63a781df8a640918ee53d537e3ee548728bacbb0" exitCode=0 Jul 23 09:55:03.453970 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:03.453939 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" event={"ID":"ef627b06-7dad-475b-b4b0-2908731467e1","Type":"ContainerDied","Data":"f50bb22fd5d91da5af824ebd63a781df8a640918ee53d537e3ee548728bacbb0"} Jul 23 09:55:03.707459 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:03.707401 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 09:55:04.211244 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:04.211141 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T09:55:03.707423362Z","UUID":"c431015a-3f06-40fb-b790-902a61f2ffd2","Handler":null,"Name":"","Endpoint":""} Jul 23 09:55:04.212556 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:04.212540 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 09:55:04.212666 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:04.212573 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 09:55:04.274517 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:04.274454 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:04.274691 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:04.274589 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:55:04.274691 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:04.274630 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:04.274691 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:04.274660 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:04.274895 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:04.274737 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:55:04.274895 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:04.274818 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:55:04.457910 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:04.457868 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bc99v" event={"ID":"000c5b44-02b0-4f5c-a70a-cce786383305","Type":"ContainerStarted","Data":"346af3ea81d37de3a35bd34d6429fa4231c906ad1a0d1b4432d5174bdbb617b5"} Jul 23 09:55:04.460292 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:04.460259 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" event={"ID":"9ec61f70-2f55-4d9e-aa93-e5110a45bb33","Type":"ContainerStarted","Data":"927cdf7bf7cda5b247783bf9b280a0d1b82d5ee2d14f0d6ae17e5bc1f559fbb4"} Jul 23 09:55:04.470981 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:04.470901 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bc99v" podStartSLOduration=5.6374032530000004 podStartE2EDuration="24.470887955s" totalImagesPullingTime="18.833484702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:04.470423545 +0000 UTC m=+24.717379811" watchObservedRunningTime="2026-07-23 09:55:04.470887955 +0000 UTC m=+24.717844221" Jul 23 09:55:04.737790 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:04.737690 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:55:05.466457 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:05.466380 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 09:55:05.466894 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:05.466808 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" event={"ID":"12d1431e-93fe-4b29-9dbc-eae156e6f45e","Type":"ContainerStarted","Data":"2c204aa9a8ec9b3fabb83e3390cbf9d469fe6b158c59688dc5bb8728f7e92c22"} Jul 23 09:55:05.469268 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:05.469240 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" event={"ID":"9ec61f70-2f55-4d9e-aa93-e5110a45bb33","Type":"ContainerStarted","Data":"9413fb583cf3dd3d5b7b193d5c87cd071d316be1a6af246360b5cae4783e2e8f"} Jul 23 09:55:05.489444 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:05.489397 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bz9rr" podStartSLOduration=4.236398823 podStartE2EDuration="25.489381703s" totalImagesPullingTime="21.25298288s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:05.488727929 +0000 UTC m=+25.735684194" watchObservedRunningTime="2026-07-23 09:55:05.489381703 +0000 UTC m=+25.736337968" Jul 23 09:55:06.023860 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:06.023814 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:55:06.024488 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:06.024457 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:55:06.274772 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:06.274459 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:06.274934 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:06.274465 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:06.274934 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:06.274459 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:06.274934 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:06.274885 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:55:06.275064 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:06.275014 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:55:06.275116 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:06.275094 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:55:06.472877 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:06.472853 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dkm2f" Jul 23 09:55:07.477491 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:07.477409 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef627b06-7dad-475b-b4b0-2908731467e1" containerID="55ffa53b54f796bdf775638cd8f02ff1445525a18f8bc0664c455e8abbd20097" exitCode=0 Jul 23 09:55:07.478343 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:07.477500 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" event={"ID":"ef627b06-7dad-475b-b4b0-2908731467e1","Type":"ContainerDied","Data":"55ffa53b54f796bdf775638cd8f02ff1445525a18f8bc0664c455e8abbd20097"} Jul 23 09:55:07.481230 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:07.481213 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 09:55:07.481668 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:07.481647 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" event={"ID":"12d1431e-93fe-4b29-9dbc-eae156e6f45e","Type":"ContainerStarted","Data":"1b20ce5d297e2d68c6f591f3e46faf0462edbd3a3e8879eceac71ae3fc4e8db1"} Jul 23 09:55:07.482126 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:07.482107 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:55:07.482243 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:07.482110 2580 scope.go:117] "RemoveContainer" containerID="2a552ee7d4159d9c9ccb129860be8d48c8cb03f31109009101f66850ae266c66" Jul 23 09:55:07.482243 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:07.482211 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:55:07.482339 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:07.482266 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:55:07.498376 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:07.498348 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:55:07.501620 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:07.501602 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:55:08.274578 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:08.274379 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:08.274782 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:08.274399 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:08.274782 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:08.274652 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:55:08.274782 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:08.274748 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:55:08.274782 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:08.274399 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:08.274978 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:08.274820 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:55:08.487100 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:08.487072 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 09:55:08.487504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:08.487388 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" event={"ID":"12d1431e-93fe-4b29-9dbc-eae156e6f45e","Type":"ContainerStarted","Data":"52a241cc2a3fbfc2b3ec20ef08e82cf658f06ea707388e014ab2229385d3e0ab"} Jul 23 09:55:08.512754 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:08.512707 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" podStartSLOduration=9.571031349 podStartE2EDuration="28.512691658s" totalImagesPullingTime="18.941660309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:08.511906605 +0000 UTC m=+28.758862870" watchObservedRunningTime="2026-07-23 09:55:08.512691658 +0000 UTC m=+28.759647923" Jul 23 09:55:09.339069 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:09.339022 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kd464"] Jul 23 09:55:09.339309 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:09.339130 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:09.339309 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:09.339236 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:55:09.339679 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:09.339659 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vw75n"] Jul 23 09:55:09.339741 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:09.339731 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:09.339816 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:09.339802 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:55:09.341634 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:09.341609 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h95kb"] Jul 23 09:55:09.341741 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:09.341694 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:09.341798 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:09.341781 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:55:09.492248 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:09.492142 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef627b06-7dad-475b-b4b0-2908731467e1" containerID="a61f773902bfe6b4f8ac8dea31c42ef4c3b58778b0313243eda127708a37a4a0" exitCode=0 Jul 23 09:55:09.492248 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:09.492219 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" event={"ID":"ef627b06-7dad-475b-b4b0-2908731467e1","Type":"ContainerDied","Data":"a61f773902bfe6b4f8ac8dea31c42ef4c3b58778b0313243eda127708a37a4a0"} Jul 23 09:55:11.274715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:11.274522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:11.275258 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:11.274522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:11.275258 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:11.274822 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:55:11.275258 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:11.274522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:11.275258 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:11.274868 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:55:11.275258 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:11.274939 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:55:11.498554 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:11.498467 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef627b06-7dad-475b-b4b0-2908731467e1" containerID="0b2b0dc22af6076d4e134dba39f0a149c719c3aab35dd8b2e71721698346be8e" exitCode=0 Jul 23 09:55:11.498554 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:11.498522 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" event={"ID":"ef627b06-7dad-475b-b4b0-2908731467e1","Type":"ContainerDied","Data":"0b2b0dc22af6076d4e134dba39f0a149c719c3aab35dd8b2e71721698346be8e"} Jul 23 09:55:13.274208 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:13.274153 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:13.274208 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:13.274180 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:13.274873 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:13.274222 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:13.274873 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:13.274300 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h95kb" podUID="ff25076a-6283-4ece-ac45-32b87379bbdd" Jul 23 09:55:13.274873 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:13.274361 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vw75n" podUID="566f324b-9a5b-4829-8ed1-69b00afb4c0d" Jul 23 09:55:13.274873 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:13.274441 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kd464" podUID="c72345fc-45c2-4b0a-9db9-593ae230265e" Jul 23 09:55:13.909944 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:13.909904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:13.909944 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:13.909949 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:13.910155 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:13.910040 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:55:13.910155 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:13.910053 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:55:13.910155 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:13.910092 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs podName:ff25076a-6283-4ece-ac45-32b87379bbdd nodeName:}" failed. No retries permitted until 2026-07-23 09:55:45.910079027 +0000 UTC m=+66.157035271 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs") pod "network-metrics-daemon-h95kb" (UID: "ff25076a-6283-4ece-ac45-32b87379bbdd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:55:13.910155 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:13.910111 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret podName:566f324b-9a5b-4829-8ed1-69b00afb4c0d nodeName:}" failed. No retries permitted until 2026-07-23 09:55:45.910097722 +0000 UTC m=+66.157053965 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret") pod "global-pull-secret-syncer-vw75n" (UID: "566f324b-9a5b-4829-8ed1-69b00afb4c0d") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:55:14.111374 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:14.111333 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg6qd\" (UniqueName: \"kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd\") pod \"network-check-target-kd464\" (UID: \"c72345fc-45c2-4b0a-9db9-593ae230265e\") " pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:14.111569 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:14.111525 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:55:14.111569 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:14.111549 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:55:14.111569 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:14.111564 2580 projected.go:196] Error preparing data for projected volume kube-api-access-sg6qd for pod openshift-network-diagnostics/network-check-target-kd464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:55:14.111726 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:14.111632 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd podName:c72345fc-45c2-4b0a-9db9-593ae230265e nodeName:}" failed. No retries permitted until 2026-07-23 09:55:46.111612543 +0000 UTC m=+66.358568788 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sg6qd" (UniqueName: "kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd") pod "network-check-target-kd464" (UID: "c72345fc-45c2-4b0a-9db9-593ae230265e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:55:15.043733 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.043647 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-55.ec2.internal" event="NodeReady" Jul 23 09:55:15.044353 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.043797 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 09:55:15.081468 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.081430 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59bb6f48dd-6zjbh"] Jul 23 09:55:15.106775 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.105931 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4csjr"] Jul 23 09:55:15.106775 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.106512 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.110587 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.109644 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 09:55:15.110587 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.109932 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vtvtf\"" Jul 23 09:55:15.110587 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.110113 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 09:55:15.110898 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.110877 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 09:55:15.120474 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.120414 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sc4hx"] Jul 23 09:55:15.120610 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.120561 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.121597 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.121576 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 09:55:15.122398 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.122381 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rdkgf\"" Jul 23 09:55:15.122484 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.122404 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 09:55:15.122544 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.122501 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 09:55:15.141938 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.141907 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59bb6f48dd-6zjbh"] Jul 23 09:55:15.142078 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.141954 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sc4hx"] Jul 23 09:55:15.142078 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.141975 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4csjr"] Jul 23 09:55:15.142078 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.142000 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sc4hx" Jul 23 09:55:15.143899 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.143871 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 09:55:15.143899 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.143887 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-96vpq\"" Jul 23 09:55:15.144069 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.143900 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 09:55:15.144069 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.144024 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 09:55:15.220615 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.220577 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-bound-sa-token\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.220615 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.220628 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cece9283-4cb5-45d7-9d63-76d5e19d15b2-ca-trust-extracted\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.220843 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.220658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3511d554-433a-49ed-87c4-fcf369aa0bf8-config-volume\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.220843 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.220744 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w6l2\" (UniqueName: \"kubernetes.io/projected/3511d554-433a-49ed-87c4-fcf369aa0bf8-kube-api-access-5w6l2\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.220843 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.220775 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-installation-pull-secrets\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.220965 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.220840 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3511d554-433a-49ed-87c4-fcf369aa0bf8-tmp-dir\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.220965 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.220866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-image-registry-private-configuration\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.220965 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.220884 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.220965 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.220904 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-certificates\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.221122 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.220974 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.221122 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.220997 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-trusted-ca\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.221122 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.221014 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtq68\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-kube-api-access-gtq68\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.274419 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.274383 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:15.274419 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.274408 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:15.274613 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.274408 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:15.277487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.276834 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 09:55:15.277487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.276834 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5t7cr\"" Jul 23 09:55:15.277487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.277112 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 09:55:15.277487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.277303 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 09:55:15.277487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.277341 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ztznw\"" Jul 23 09:55:15.277487 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.277405 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 09:55:15.322330 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.322300 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3511d554-433a-49ed-87c4-fcf369aa0bf8-tmp-dir\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.322506 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.322336 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-image-registry-private-configuration\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.322506 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.322366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.322506 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.322392 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-certificates\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.322506 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.322427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.322506 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.322450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-trusted-ca\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.322506 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.322469 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33558acb-61b9-4b54-9c54-4ef58db51b48-cert\") pod \"ingress-canary-sc4hx\" (UID: \"33558acb-61b9-4b54-9c54-4ef58db51b48\") " pod="openshift-ingress-canary/ingress-canary-sc4hx" Jul 23 09:55:15.322506 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.322487 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtq68\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-kube-api-access-gtq68\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.322506 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:15.322499 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 09:55:15.322843 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:15.322521 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59bb6f48dd-6zjbh: secret "image-registry-tls" not found Jul 23 09:55:15.322843 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:15.322599 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls podName:cece9283-4cb5-45d7-9d63-76d5e19d15b2 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:15.822573571 +0000 UTC m=+36.069529828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls") pod "image-registry-59bb6f48dd-6zjbh" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2") : secret "image-registry-tls" not found Jul 23 09:55:15.322843 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.322695 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3511d554-433a-49ed-87c4-fcf369aa0bf8-tmp-dir\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.323640 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.322507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pfmz\" (UniqueName: \"kubernetes.io/projected/33558acb-61b9-4b54-9c54-4ef58db51b48-kube-api-access-9pfmz\") pod \"ingress-canary-sc4hx\" (UID: \"33558acb-61b9-4b54-9c54-4ef58db51b48\") " pod="openshift-ingress-canary/ingress-canary-sc4hx" Jul 23 09:55:15.323640 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.323043 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-bound-sa-token\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.323640 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.323075 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cece9283-4cb5-45d7-9d63-76d5e19d15b2-ca-trust-extracted\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.323640 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:15.323080 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 09:55:15.323640 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.323104 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3511d554-433a-49ed-87c4-fcf369aa0bf8-config-volume\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.323640 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.323161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w6l2\" (UniqueName: \"kubernetes.io/projected/3511d554-433a-49ed-87c4-fcf369aa0bf8-kube-api-access-5w6l2\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.323640 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.323212 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-installation-pull-secrets\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.323640 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.323279 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-certificates\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.323640 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:15.323344 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls podName:3511d554-433a-49ed-87c4-fcf369aa0bf8 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:15.823319975 +0000 UTC m=+36.070276220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls") pod "dns-default-4csjr" (UID: "3511d554-433a-49ed-87c4-fcf369aa0bf8") : secret "dns-default-metrics-tls" not found Jul 23 09:55:15.323640 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.323440 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cece9283-4cb5-45d7-9d63-76d5e19d15b2-ca-trust-extracted\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.324108 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.323843 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3511d554-433a-49ed-87c4-fcf369aa0bf8-config-volume\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.324221 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.324174 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-trusted-ca\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.327226 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.327168 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-image-registry-private-configuration\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.327337 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.327238 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-installation-pull-secrets\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.333691 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.333666 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtq68\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-kube-api-access-gtq68\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.334334 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.334044 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-bound-sa-token\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.334825 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.334794 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w6l2\" (UniqueName: \"kubernetes.io/projected/3511d554-433a-49ed-87c4-fcf369aa0bf8-kube-api-access-5w6l2\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.424604 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.424568 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33558acb-61b9-4b54-9c54-4ef58db51b48-cert\") pod \"ingress-canary-sc4hx\" (UID: \"33558acb-61b9-4b54-9c54-4ef58db51b48\") " pod="openshift-ingress-canary/ingress-canary-sc4hx" Jul 23 09:55:15.424604 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.424611 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pfmz\" (UniqueName: \"kubernetes.io/projected/33558acb-61b9-4b54-9c54-4ef58db51b48-kube-api-access-9pfmz\") pod \"ingress-canary-sc4hx\" (UID: \"33558acb-61b9-4b54-9c54-4ef58db51b48\") " pod="openshift-ingress-canary/ingress-canary-sc4hx" Jul 23 09:55:15.427048 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.427017 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33558acb-61b9-4b54-9c54-4ef58db51b48-cert\") pod \"ingress-canary-sc4hx\" (UID: \"33558acb-61b9-4b54-9c54-4ef58db51b48\") " pod="openshift-ingress-canary/ingress-canary-sc4hx" Jul 23 09:55:15.431984 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.431952 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pfmz\" (UniqueName: \"kubernetes.io/projected/33558acb-61b9-4b54-9c54-4ef58db51b48-kube-api-access-9pfmz\") pod \"ingress-canary-sc4hx\" (UID: \"33558acb-61b9-4b54-9c54-4ef58db51b48\") " pod="openshift-ingress-canary/ingress-canary-sc4hx" Jul 23 09:55:15.451961 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.451932 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sc4hx" Jul 23 09:55:15.685862 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.685685 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sc4hx"] Jul 23 09:55:15.747722 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:55:15.747683 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33558acb_61b9_4b54_9c54_4ef58db51b48.slice/crio-a33e14f5d5c5096b29165c480e71f7136a240ff0617013d6031ffb7b7a1f323d WatchSource:0}: Error finding container a33e14f5d5c5096b29165c480e71f7136a240ff0617013d6031ffb7b7a1f323d: Status 404 returned error can't find the container with id a33e14f5d5c5096b29165c480e71f7136a240ff0617013d6031ffb7b7a1f323d Jul 23 09:55:15.827122 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.827099 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:15.827234 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:15.827133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:15.827272 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:15.827261 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 09:55:15.827317 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:15.827273 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 09:55:15.827317 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:15.827293 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59bb6f48dd-6zjbh: secret "image-registry-tls" not found Jul 23 09:55:15.827317 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:15.827310 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls podName:3511d554-433a-49ed-87c4-fcf369aa0bf8 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:16.827297353 +0000 UTC m=+37.074253596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls") pod "dns-default-4csjr" (UID: "3511d554-433a-49ed-87c4-fcf369aa0bf8") : secret "dns-default-metrics-tls" not found Jul 23 09:55:15.827433 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:15.827342 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls podName:cece9283-4cb5-45d7-9d63-76d5e19d15b2 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:16.827325716 +0000 UTC m=+37.074281959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls") pod "image-registry-59bb6f48dd-6zjbh" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2") : secret "image-registry-tls" not found Jul 23 09:55:16.513540 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:16.513506 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef627b06-7dad-475b-b4b0-2908731467e1" containerID="7f5a3aec93181ccf8ceca50db6a1d65a65dd38beb08fa3604ffa60fb3db16634" exitCode=0 Jul 23 09:55:16.514114 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:16.513585 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" event={"ID":"ef627b06-7dad-475b-b4b0-2908731467e1","Type":"ContainerDied","Data":"7f5a3aec93181ccf8ceca50db6a1d65a65dd38beb08fa3604ffa60fb3db16634"} Jul 23 09:55:16.514816 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:16.514779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sc4hx" event={"ID":"33558acb-61b9-4b54-9c54-4ef58db51b48","Type":"ContainerStarted","Data":"a33e14f5d5c5096b29165c480e71f7136a240ff0617013d6031ffb7b7a1f323d"} Jul 23 09:55:16.836409 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:16.836309 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:16.836409 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:16.836358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:16.836624 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:16.836474 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 09:55:16.836624 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:16.836481 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 09:55:16.836624 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:16.836495 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59bb6f48dd-6zjbh: secret "image-registry-tls" not found Jul 23 09:55:16.836624 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:16.836554 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls podName:cece9283-4cb5-45d7-9d63-76d5e19d15b2 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:18.836534441 +0000 UTC m=+39.083490691 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls") pod "image-registry-59bb6f48dd-6zjbh" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2") : secret "image-registry-tls" not found Jul 23 09:55:16.836624 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:16.836571 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls podName:3511d554-433a-49ed-87c4-fcf369aa0bf8 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:18.836562971 +0000 UTC m=+39.083519220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls") pod "dns-default-4csjr" (UID: "3511d554-433a-49ed-87c4-fcf369aa0bf8") : secret "dns-default-metrics-tls" not found Jul 23 09:55:17.520810 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:17.520763 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef627b06-7dad-475b-b4b0-2908731467e1" containerID="49927ac218322c804918b61ca257a6db955a612e455a8b9d576b4a4b66bd51e9" exitCode=0 Jul 23 09:55:17.521318 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:17.520839 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" event={"ID":"ef627b06-7dad-475b-b4b0-2908731467e1","Type":"ContainerDied","Data":"49927ac218322c804918b61ca257a6db955a612e455a8b9d576b4a4b66bd51e9"} Jul 23 09:55:18.527142 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:18.526911 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" event={"ID":"ef627b06-7dad-475b-b4b0-2908731467e1","Type":"ContainerStarted","Data":"cef8f27f6aa41bce9d2553e68d7e32e14474d11bafcddcb8eaac6524f9f9f766"} Jul 23 09:55:18.528397 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:18.528374 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sc4hx" event={"ID":"33558acb-61b9-4b54-9c54-4ef58db51b48","Type":"ContainerStarted","Data":"9e6df80d708accdb2529a0d0b22298e4a61b46ce9d8dd1a5f524b467cbcdccee"} Jul 23 09:55:18.547692 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:18.547639 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h9k5p" podStartSLOduration=8.522738933 podStartE2EDuration="38.54762514s" totalImagesPullingTime="30.024886207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:18.546604341 +0000 UTC m=+38.793560607" watchObservedRunningTime="2026-07-23 09:55:18.54762514 +0000 UTC m=+38.794581411" Jul 23 09:55:18.563333 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:18.563283 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sc4hx" podStartSLOduration=1.48412702 podStartE2EDuration="3.563264968s" totalImagesPullingTime="2.079137948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:18.56239903 +0000 UTC m=+38.809355295" watchObservedRunningTime="2026-07-23 09:55:18.563264968 +0000 UTC m=+38.810221232" Jul 23 09:55:18.852297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:18.852210 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:18.852297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:18.852260 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:18.852472 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:18.852295 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 09:55:18.852472 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:18.852333 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59bb6f48dd-6zjbh: secret "image-registry-tls" not found Jul 23 09:55:18.852472 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:18.852385 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 09:55:18.852472 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:18.852391 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls podName:cece9283-4cb5-45d7-9d63-76d5e19d15b2 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:22.852373416 +0000 UTC m=+43.099329660 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls") pod "image-registry-59bb6f48dd-6zjbh" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2") : secret "image-registry-tls" not found Jul 23 09:55:18.852472 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:18.852431 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls podName:3511d554-433a-49ed-87c4-fcf369aa0bf8 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:22.852419796 +0000 UTC m=+43.099376041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls") pod "dns-default-4csjr" (UID: "3511d554-433a-49ed-87c4-fcf369aa0bf8") : secret "dns-default-metrics-tls" not found Jul 23 09:55:22.576786 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.576752 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rn7mv"] Jul 23 09:55:22.621670 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.621626 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rn7mv"] Jul 23 09:55:22.621859 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.621774 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.624016 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.623994 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 09:55:22.624165 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.623995 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 09:55:22.624503 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.624474 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 09:55:22.624566 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.624474 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-96kbl\"" Jul 23 09:55:22.624566 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.624475 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 09:55:22.680907 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.680873 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4cb72c19-cc93-4b00-b759-464689c41963-crio-socket\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.680907 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.680909 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4cb72c19-cc93-4b00-b759-464689c41963-data-volume\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.681131 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.680929 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.681131 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.680950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.681131 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.681079 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgtlp\" (UniqueName: \"kubernetes.io/projected/4cb72c19-cc93-4b00-b759-464689c41963-kube-api-access-qgtlp\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.782186 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.782142 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4cb72c19-cc93-4b00-b759-464689c41963-crio-socket\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.782186 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.782182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4cb72c19-cc93-4b00-b759-464689c41963-data-volume\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.782186 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.782218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.782483 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.782237 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.782483 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.782303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgtlp\" (UniqueName: \"kubernetes.io/projected/4cb72c19-cc93-4b00-b759-464689c41963-kube-api-access-qgtlp\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.782483 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:22.782360 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 09:55:22.782483 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:22.782380 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 09:55:22.782483 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.782377 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4cb72c19-cc93-4b00-b759-464689c41963-crio-socket\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.782483 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:22.782443 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm podName:4cb72c19-cc93-4b00-b759-464689c41963 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:23.282422792 +0000 UTC m=+43.529379035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rn7mv" (UID: "4cb72c19-cc93-4b00-b759-464689c41963") : configmap "kube-rbac-proxy" not found Jul 23 09:55:22.782483 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:22.782463 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls podName:4cb72c19-cc93-4b00-b759-464689c41963 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:23.282456038 +0000 UTC m=+43.529412282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rn7mv" (UID: "4cb72c19-cc93-4b00-b759-464689c41963") : secret "insights-runtime-extractor-tls" not found Jul 23 09:55:22.782883 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.782563 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4cb72c19-cc93-4b00-b759-464689c41963-data-volume\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.795999 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.795965 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgtlp\" (UniqueName: \"kubernetes.io/projected/4cb72c19-cc93-4b00-b759-464689c41963-kube-api-access-qgtlp\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:22.883212 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.883083 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:22.883212 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:22.883144 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:22.883456 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:22.883278 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 09:55:22.883456 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:22.883280 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 09:55:22.883456 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:22.883306 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59bb6f48dd-6zjbh: secret "image-registry-tls" not found Jul 23 09:55:22.883456 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:22.883352 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls podName:3511d554-433a-49ed-87c4-fcf369aa0bf8 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:30.883333618 +0000 UTC m=+51.130289865 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls") pod "dns-default-4csjr" (UID: "3511d554-433a-49ed-87c4-fcf369aa0bf8") : secret "dns-default-metrics-tls" not found Jul 23 09:55:22.883456 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:22.883415 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls podName:cece9283-4cb5-45d7-9d63-76d5e19d15b2 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:30.883388633 +0000 UTC m=+51.130344884 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls") pod "image-registry-59bb6f48dd-6zjbh" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2") : secret "image-registry-tls" not found Jul 23 09:55:23.285803 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:23.285766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:23.285803 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:23.285807 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:23.286075 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:23.285925 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 09:55:23.286075 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:23.285944 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 09:55:23.286075 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:23.285994 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm podName:4cb72c19-cc93-4b00-b759-464689c41963 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:24.285978537 +0000 UTC m=+44.532934782 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rn7mv" (UID: "4cb72c19-cc93-4b00-b759-464689c41963") : configmap "kube-rbac-proxy" not found Jul 23 09:55:23.286075 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:23.286008 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls podName:4cb72c19-cc93-4b00-b759-464689c41963 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:24.286002548 +0000 UTC m=+44.532958792 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rn7mv" (UID: "4cb72c19-cc93-4b00-b759-464689c41963") : secret "insights-runtime-extractor-tls" not found Jul 23 09:55:24.293541 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:24.293498 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:24.293541 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:24.293540 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:24.293960 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:24.293611 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 09:55:24.293960 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:24.293663 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 09:55:24.293960 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:24.293676 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm podName:4cb72c19-cc93-4b00-b759-464689c41963 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:26.293657798 +0000 UTC m=+46.540614044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rn7mv" (UID: "4cb72c19-cc93-4b00-b759-464689c41963") : configmap "kube-rbac-proxy" not found Jul 23 09:55:24.293960 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:24.293717 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls podName:4cb72c19-cc93-4b00-b759-464689c41963 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:26.293703161 +0000 UTC m=+46.540659405 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rn7mv" (UID: "4cb72c19-cc93-4b00-b759-464689c41963") : secret "insights-runtime-extractor-tls" not found Jul 23 09:55:24.561452 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:24.561376 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lcx56_4a666fcc-084b-4948-8fac-8b7824ed1b62/dns-node-resolver/0.log" Jul 23 09:55:25.561968 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:25.561940 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kbzkh_c83bbb36-0921-4e7e-a036-c6fad9d60e43/node-ca/0.log" Jul 23 09:55:26.308599 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:26.308558 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:26.308599 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:26.308598 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:26.308822 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:26.308683 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 09:55:26.308822 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:26.308715 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 09:55:26.308822 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:26.308736 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm podName:4cb72c19-cc93-4b00-b759-464689c41963 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:30.308723049 +0000 UTC m=+50.555679293 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rn7mv" (UID: "4cb72c19-cc93-4b00-b759-464689c41963") : configmap "kube-rbac-proxy" not found Jul 23 09:55:26.308822 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:26.308767 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls podName:4cb72c19-cc93-4b00-b759-464689c41963 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:30.308754143 +0000 UTC m=+50.555710386 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rn7mv" (UID: "4cb72c19-cc93-4b00-b759-464689c41963") : secret "insights-runtime-extractor-tls" not found Jul 23 09:55:26.362634 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:26.362606 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sc4hx_33558acb-61b9-4b54-9c54-4ef58db51b48/serve-healthcheck-canary/0.log" Jul 23 09:55:30.340817 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:30.340770 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:30.340817 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:30.340815 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:30.341475 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:30.340920 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 09:55:30.341475 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:30.340998 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls podName:4cb72c19-cc93-4b00-b759-464689c41963 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:38.340980433 +0000 UTC m=+58.587936691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rn7mv" (UID: "4cb72c19-cc93-4b00-b759-464689c41963") : secret "insights-runtime-extractor-tls" not found Jul 23 09:55:30.341475 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:30.341326 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4cb72c19-cc93-4b00-b759-464689c41963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:30.947842 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:30.947803 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:30.947842 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:30.947846 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:30.948100 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:30.947949 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 09:55:30.948100 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:30.947959 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 09:55:30.948100 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:30.947980 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59bb6f48dd-6zjbh: secret "image-registry-tls" not found Jul 23 09:55:30.948100 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:30.948005 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls podName:3511d554-433a-49ed-87c4-fcf369aa0bf8 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:46.947990026 +0000 UTC m=+67.194946273 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls") pod "dns-default-4csjr" (UID: "3511d554-433a-49ed-87c4-fcf369aa0bf8") : secret "dns-default-metrics-tls" not found Jul 23 09:55:30.948100 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:30.948047 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls podName:cece9283-4cb5-45d7-9d63-76d5e19d15b2 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:46.948029988 +0000 UTC m=+67.194986247 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls") pod "image-registry-59bb6f48dd-6zjbh" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2") : secret "image-registry-tls" not found Jul 23 09:55:38.406261 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:38.406217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:38.408606 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:38.408586 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4cb72c19-cc93-4b00-b759-464689c41963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rn7mv\" (UID: \"4cb72c19-cc93-4b00-b759-464689c41963\") " pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:38.530816 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:38.530780 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rn7mv" Jul 23 09:55:38.672354 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:38.672276 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rn7mv"] Jul 23 09:55:38.676111 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:55:38.676078 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb72c19_cc93_4b00_b759_464689c41963.slice/crio-c38e5de5ad006620e6b02a7aac2478834e929fd49dbdef78b2a381eec8b6aa5e WatchSource:0}: Error finding container c38e5de5ad006620e6b02a7aac2478834e929fd49dbdef78b2a381eec8b6aa5e: Status 404 returned error can't find the container with id c38e5de5ad006620e6b02a7aac2478834e929fd49dbdef78b2a381eec8b6aa5e Jul 23 09:55:39.506058 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:39.506025 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q4p8r" Jul 23 09:55:39.580614 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:39.580574 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rn7mv" event={"ID":"4cb72c19-cc93-4b00-b759-464689c41963","Type":"ContainerStarted","Data":"c38e5de5ad006620e6b02a7aac2478834e929fd49dbdef78b2a381eec8b6aa5e"} Jul 23 09:55:44.594883 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:44.594845 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rn7mv" event={"ID":"4cb72c19-cc93-4b00-b759-464689c41963","Type":"ContainerStarted","Data":"0018b1395d00ab1209a8534335ed9e7acb7bbe64a05b3e32695b22adcc056028"} Jul 23 09:55:45.600586 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.600553 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h"] Jul 23 09:55:45.609491 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.609459 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s"] Jul 23 09:55:45.609679 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.609570 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h" Jul 23 09:55:45.612069 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.612046 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:45.612628 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.612605 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 23 09:55:45.612823 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.612610 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 23 09:55:45.612925 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.612896 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 23 09:55:45.613158 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.613140 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 23 09:55:45.613297 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.613277 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-c8xfh\"" Jul 23 09:55:45.613889 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.613866 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 23 09:55:45.620369 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.620341 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s"] Jul 23 09:55:45.621334 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.621310 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h"] Jul 23 09:55:45.672301 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.672249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/087b3861-7635-4c06-97f9-1226e272dc8e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-777455499d-f6d9h\" (UID: \"087b3861-7635-4c06-97f9-1226e272dc8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h" Jul 23 09:55:45.672485 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.672311 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpj6k\" (UniqueName: \"kubernetes.io/projected/087b3861-7635-4c06-97f9-1226e272dc8e-kube-api-access-kpj6k\") pod \"managed-serviceaccount-addon-agent-777455499d-f6d9h\" (UID: \"087b3861-7635-4c06-97f9-1226e272dc8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h" Jul 23 09:55:45.672485 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.672355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/811ec163-29b4-44c8-83a2-ee6225cbfbf8-tmp\") pod \"klusterlet-addon-workmgr-cc8d5965d-48c6s\" (UID: \"811ec163-29b4-44c8-83a2-ee6225cbfbf8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:45.672485 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.672452 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/811ec163-29b4-44c8-83a2-ee6225cbfbf8-klusterlet-config\") pod \"klusterlet-addon-workmgr-cc8d5965d-48c6s\" (UID: \"811ec163-29b4-44c8-83a2-ee6225cbfbf8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:45.672632 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.672602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klq5t\" (UniqueName: \"kubernetes.io/projected/811ec163-29b4-44c8-83a2-ee6225cbfbf8-kube-api-access-klq5t\") pod \"klusterlet-addon-workmgr-cc8d5965d-48c6s\" (UID: \"811ec163-29b4-44c8-83a2-ee6225cbfbf8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:45.773730 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.773684 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/811ec163-29b4-44c8-83a2-ee6225cbfbf8-tmp\") pod \"klusterlet-addon-workmgr-cc8d5965d-48c6s\" (UID: \"811ec163-29b4-44c8-83a2-ee6225cbfbf8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:45.773917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.773782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/811ec163-29b4-44c8-83a2-ee6225cbfbf8-klusterlet-config\") pod \"klusterlet-addon-workmgr-cc8d5965d-48c6s\" (UID: \"811ec163-29b4-44c8-83a2-ee6225cbfbf8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:45.773917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.773844 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klq5t\" (UniqueName: \"kubernetes.io/projected/811ec163-29b4-44c8-83a2-ee6225cbfbf8-kube-api-access-klq5t\") pod \"klusterlet-addon-workmgr-cc8d5965d-48c6s\" (UID: \"811ec163-29b4-44c8-83a2-ee6225cbfbf8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:45.773917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.773879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/087b3861-7635-4c06-97f9-1226e272dc8e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-777455499d-f6d9h\" (UID: \"087b3861-7635-4c06-97f9-1226e272dc8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h" Jul 23 09:55:45.773917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.773905 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpj6k\" (UniqueName: \"kubernetes.io/projected/087b3861-7635-4c06-97f9-1226e272dc8e-kube-api-access-kpj6k\") pod \"managed-serviceaccount-addon-agent-777455499d-f6d9h\" (UID: \"087b3861-7635-4c06-97f9-1226e272dc8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h" Jul 23 09:55:45.774172 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.774140 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/811ec163-29b4-44c8-83a2-ee6225cbfbf8-tmp\") pod \"klusterlet-addon-workmgr-cc8d5965d-48c6s\" (UID: \"811ec163-29b4-44c8-83a2-ee6225cbfbf8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:45.776710 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.776679 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/087b3861-7635-4c06-97f9-1226e272dc8e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-777455499d-f6d9h\" (UID: \"087b3861-7635-4c06-97f9-1226e272dc8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h" Jul 23 09:55:45.776858 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.776836 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/811ec163-29b4-44c8-83a2-ee6225cbfbf8-klusterlet-config\") pod \"klusterlet-addon-workmgr-cc8d5965d-48c6s\" (UID: \"811ec163-29b4-44c8-83a2-ee6225cbfbf8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:45.785757 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.785732 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klq5t\" (UniqueName: \"kubernetes.io/projected/811ec163-29b4-44c8-83a2-ee6225cbfbf8-kube-api-access-klq5t\") pod \"klusterlet-addon-workmgr-cc8d5965d-48c6s\" (UID: \"811ec163-29b4-44c8-83a2-ee6225cbfbf8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:45.787636 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.787612 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpj6k\" (UniqueName: \"kubernetes.io/projected/087b3861-7635-4c06-97f9-1226e272dc8e-kube-api-access-kpj6k\") pod \"managed-serviceaccount-addon-agent-777455499d-f6d9h\" (UID: \"087b3861-7635-4c06-97f9-1226e272dc8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h" Jul 23 09:55:45.945564 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.945474 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h" Jul 23 09:55:45.955550 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.955519 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:45.975820 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.975784 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:45.975991 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.975831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:45.977920 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.977894 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 09:55:45.978056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.977951 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 09:55:45.989460 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.989430 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/566f324b-9a5b-4829-8ed1-69b00afb4c0d-original-pull-secret\") pod \"global-pull-secret-syncer-vw75n\" (UID: \"566f324b-9a5b-4829-8ed1-69b00afb4c0d\") " pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:45.989590 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:45.989537 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff25076a-6283-4ece-ac45-32b87379bbdd-metrics-certs\") pod \"network-metrics-daemon-h95kb\" (UID: \"ff25076a-6283-4ece-ac45-32b87379bbdd\") " pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:46.177357 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.177322 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg6qd\" (UniqueName: \"kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd\") pod \"network-check-target-kd464\" (UID: \"c72345fc-45c2-4b0a-9db9-593ae230265e\") " pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:46.179430 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.179409 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 09:55:46.187749 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.187724 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ztznw\"" Jul 23 09:55:46.189309 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.189292 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 09:55:46.196706 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.196643 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h95kb" Jul 23 09:55:46.197821 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.197792 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vw75n" Jul 23 09:55:46.200653 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.200627 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg6qd\" (UniqueName: \"kubernetes.io/projected/c72345fc-45c2-4b0a-9db9-593ae230265e-kube-api-access-sg6qd\") pod \"network-check-target-kd464\" (UID: \"c72345fc-45c2-4b0a-9db9-593ae230265e\") " pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:46.494321 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.494289 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5t7cr\"" Jul 23 09:55:46.502942 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.502905 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:46.986064 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.984903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:46.986064 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.984955 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:46.991397 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.990431 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls\") pod \"image-registry-59bb6f48dd-6zjbh\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:46.991397 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:46.990634 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3511d554-433a-49ed-87c4-fcf369aa0bf8-metrics-tls\") pod \"dns-default-4csjr\" (UID: \"3511d554-433a-49ed-87c4-fcf369aa0bf8\") " pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:47.075368 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.075335 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h"] Jul 23 09:55:47.078750 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:55:47.078716 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod087b3861_7635_4c06_97f9_1226e272dc8e.slice/crio-f4b3d9f55e8cf79b00020257668357856aa062dac36246eaa9999c552f9c8981 WatchSource:0}: Error finding container f4b3d9f55e8cf79b00020257668357856aa062dac36246eaa9999c552f9c8981: Status 404 returned error can't find the container with id f4b3d9f55e8cf79b00020257668357856aa062dac36246eaa9999c552f9c8981 Jul 23 09:55:47.222441 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.222415 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vtvtf\"" Jul 23 09:55:47.230992 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.230959 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:47.231672 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.231652 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rdkgf\"" Jul 23 09:55:47.240436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.240408 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:47.299626 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.299588 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vw75n"] Jul 23 09:55:47.304430 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:55:47.304373 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod566f324b_9a5b_4829_8ed1_69b00afb4c0d.slice/crio-3529dd09fb2d8d6e27d02ae1d16837f41ec2eeaeb598caaafb63826f2994c5ae WatchSource:0}: Error finding container 3529dd09fb2d8d6e27d02ae1d16837f41ec2eeaeb598caaafb63826f2994c5ae: Status 404 returned error can't find the container with id 3529dd09fb2d8d6e27d02ae1d16837f41ec2eeaeb598caaafb63826f2994c5ae Jul 23 09:55:47.307676 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.305853 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kd464"] Jul 23 09:55:47.308077 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.308010 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s"] Jul 23 09:55:47.309956 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.309106 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h95kb"] Jul 23 09:55:47.310178 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:55:47.309970 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72345fc_45c2_4b0a_9db9_593ae230265e.slice/crio-5b929fc89b72c30714a945cf305a7bacc7dd44a31304fcabcf69f6e1beacced6 WatchSource:0}: Error finding container 5b929fc89b72c30714a945cf305a7bacc7dd44a31304fcabcf69f6e1beacced6: Status 404 returned error can't find the container with id 5b929fc89b72c30714a945cf305a7bacc7dd44a31304fcabcf69f6e1beacced6 Jul 23 09:55:47.312342 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:55:47.312312 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff25076a_6283_4ece_ac45_32b87379bbdd.slice/crio-347b17c8929097663976cf287901d07289b260f15f9d3c7f2891037d19511790 WatchSource:0}: Error finding container 347b17c8929097663976cf287901d07289b260f15f9d3c7f2891037d19511790: Status 404 returned error can't find the container with id 347b17c8929097663976cf287901d07289b260f15f9d3c7f2891037d19511790 Jul 23 09:55:47.313596 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:55:47.313558 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811ec163_29b4_44c8_83a2_ee6225cbfbf8.slice/crio-a8e37e1991749d2d842a65a5cf203d65f990b6b448b5e8953d348e0d1b0b31b0 WatchSource:0}: Error finding container a8e37e1991749d2d842a65a5cf203d65f990b6b448b5e8953d348e0d1b0b31b0: Status 404 returned error can't find the container with id a8e37e1991749d2d842a65a5cf203d65f990b6b448b5e8953d348e0d1b0b31b0 Jul 23 09:55:47.379471 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.379397 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59bb6f48dd-6zjbh"] Jul 23 09:55:47.382751 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:55:47.382719 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcece9283_4cb5_45d7_9d63_76d5e19d15b2.slice/crio-8ae94208efaf0af9d356b20b328de458699678ed396bf10dabdcbe40993e5e0a WatchSource:0}: Error finding container 8ae94208efaf0af9d356b20b328de458699678ed396bf10dabdcbe40993e5e0a: Status 404 returned error can't find the container with id 8ae94208efaf0af9d356b20b328de458699678ed396bf10dabdcbe40993e5e0a Jul 23 09:55:47.392570 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.392533 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4csjr"] Jul 23 09:55:47.396417 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:55:47.396389 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3511d554_433a_49ed_87c4_fcf369aa0bf8.slice/crio-4186389f1050834bb4892a4b985f28cd67a2ff3602e158cab94dee1789383590 WatchSource:0}: Error finding container 4186389f1050834bb4892a4b985f28cd67a2ff3602e158cab94dee1789383590: Status 404 returned error can't find the container with id 4186389f1050834bb4892a4b985f28cd67a2ff3602e158cab94dee1789383590 Jul 23 09:55:47.605853 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.605753 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" event={"ID":"811ec163-29b4-44c8-83a2-ee6225cbfbf8","Type":"ContainerStarted","Data":"a8e37e1991749d2d842a65a5cf203d65f990b6b448b5e8953d348e0d1b0b31b0"} Jul 23 09:55:47.607180 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.607153 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" event={"ID":"cece9283-4cb5-45d7-9d63-76d5e19d15b2","Type":"ContainerStarted","Data":"58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8"} Jul 23 09:55:47.607402 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.607215 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" event={"ID":"cece9283-4cb5-45d7-9d63-76d5e19d15b2","Type":"ContainerStarted","Data":"8ae94208efaf0af9d356b20b328de458699678ed396bf10dabdcbe40993e5e0a"} Jul 23 09:55:47.608598 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.608570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h95kb" event={"ID":"ff25076a-6283-4ece-ac45-32b87379bbdd","Type":"ContainerStarted","Data":"347b17c8929097663976cf287901d07289b260f15f9d3c7f2891037d19511790"} Jul 23 09:55:47.609865 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.609842 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vw75n" event={"ID":"566f324b-9a5b-4829-8ed1-69b00afb4c0d","Type":"ContainerStarted","Data":"3529dd09fb2d8d6e27d02ae1d16837f41ec2eeaeb598caaafb63826f2994c5ae"} Jul 23 09:55:47.611794 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.611763 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rn7mv" event={"ID":"4cb72c19-cc93-4b00-b759-464689c41963","Type":"ContainerStarted","Data":"9073b5c9ae9bdc1ece1ece706ddd6982e7a7abe45aa57b3f3dd622fcfcb0108c"} Jul 23 09:55:47.612918 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.612899 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4csjr" event={"ID":"3511d554-433a-49ed-87c4-fcf369aa0bf8","Type":"ContainerStarted","Data":"4186389f1050834bb4892a4b985f28cd67a2ff3602e158cab94dee1789383590"} Jul 23 09:55:47.613993 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.613975 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kd464" event={"ID":"c72345fc-45c2-4b0a-9db9-593ae230265e","Type":"ContainerStarted","Data":"5b929fc89b72c30714a945cf305a7bacc7dd44a31304fcabcf69f6e1beacced6"} Jul 23 09:55:47.615070 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.615045 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h" event={"ID":"087b3861-7635-4c06-97f9-1226e272dc8e","Type":"ContainerStarted","Data":"f4b3d9f55e8cf79b00020257668357856aa062dac36246eaa9999c552f9c8981"} Jul 23 09:55:47.624945 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:47.624892 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" podStartSLOduration=58.624874908 podStartE2EDuration="58.624874908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:47.624652328 +0000 UTC m=+67.871608593" watchObservedRunningTime="2026-07-23 09:55:47.624874908 +0000 UTC m=+67.871831172" Jul 23 09:55:48.634170 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:48.632400 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:55:49.643910 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:49.643868 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h95kb" event={"ID":"ff25076a-6283-4ece-ac45-32b87379bbdd","Type":"ContainerStarted","Data":"58c026ab02e5d280f341ae45638a48d0e50824dc2b23b1eb8de75f06fb9de769"} Jul 23 09:55:49.644394 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:49.643935 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h95kb" event={"ID":"ff25076a-6283-4ece-ac45-32b87379bbdd","Type":"ContainerStarted","Data":"d7753b1e10a6beb8a068f4c6dac737353dca9f7729288274d96952fd878a335f"} Jul 23 09:55:53.374233 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:53.374151 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-h95kb" podStartSLOduration=71.825934934 podStartE2EDuration="1m13.374127284s" totalImagesPullingTime="1.54819235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:49.664012837 +0000 UTC m=+69.910969104" watchObservedRunningTime="2026-07-23 09:55:53.374127284 +0000 UTC m=+73.621083552" Jul 23 09:55:53.374686 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:53.374581 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8"] Jul 23 09:55:53.391861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:53.391832 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8"] Jul 23 09:55:53.392005 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:53.391958 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8" Jul 23 09:55:53.394210 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:53.394173 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 23 09:55:53.394361 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:53.394344 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cmz8m\"" Jul 23 09:55:53.541132 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:53.541097 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9697c23f-7286-4092-86c3-41b8fb851d67-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-8n7z8\" (UID: \"9697c23f-7286-4092-86c3-41b8fb851d67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8" Jul 23 09:55:53.642235 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:53.642122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9697c23f-7286-4092-86c3-41b8fb851d67-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-8n7z8\" (UID: \"9697c23f-7286-4092-86c3-41b8fb851d67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8" Jul 23 09:55:53.642384 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:53.642306 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 23 09:55:53.642433 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:55:53.642390 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9697c23f-7286-4092-86c3-41b8fb851d67-tls-certificates podName:9697c23f-7286-4092-86c3-41b8fb851d67 nodeName:}" failed. No retries permitted until 2026-07-23 09:55:54.142368974 +0000 UTC m=+74.389325222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/9697c23f-7286-4092-86c3-41b8fb851d67-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-8n7z8" (UID: "9697c23f-7286-4092-86c3-41b8fb851d67") : secret "prometheus-operator-admission-webhook-tls" not found Jul 23 09:55:54.146247 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:54.146203 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9697c23f-7286-4092-86c3-41b8fb851d67-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-8n7z8\" (UID: \"9697c23f-7286-4092-86c3-41b8fb851d67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8" Jul 23 09:55:54.149431 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:54.149402 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9697c23f-7286-4092-86c3-41b8fb851d67-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-8n7z8\" (UID: \"9697c23f-7286-4092-86c3-41b8fb851d67\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8" Jul 23 09:55:54.302494 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:54.302457 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8" Jul 23 09:55:57.692457 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:57.691873 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rn7mv" event={"ID":"4cb72c19-cc93-4b00-b759-464689c41963","Type":"ContainerStarted","Data":"6f177f0847e16ed3349b497918a5fcbb9d3de8a31549dad06677f23037f45f75"} Jul 23 09:55:57.711029 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:57.710902 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rn7mv" podStartSLOduration=16.997487423 podStartE2EDuration="35.710881219s" totalImagesPullingTime="18.713393796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:55:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:57.709686802 +0000 UTC m=+77.956643069" watchObservedRunningTime="2026-07-23 09:55:57.710881219 +0000 UTC m=+77.957837486" Jul 23 09:55:57.731624 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:57.730504 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8"] Jul 23 09:55:58.697272 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.697228 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vw75n" event={"ID":"566f324b-9a5b-4829-8ed1-69b00afb4c0d","Type":"ContainerStarted","Data":"df6aa129e217b42c4345867d72a981cc089c37298cd56d73cc238de8069a4902"} Jul 23 09:55:58.698508 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.698476 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8" event={"ID":"9697c23f-7286-4092-86c3-41b8fb851d67","Type":"ContainerStarted","Data":"0dbfcac71f230a79d28cec5e7728c9513b50259409b0400fd9d9e2c35ccb4e7b"} Jul 23 09:55:58.700578 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.700547 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4csjr" event={"ID":"3511d554-433a-49ed-87c4-fcf369aa0bf8","Type":"ContainerStarted","Data":"a163bebf03bc39da1f498f60973e100d450fa31033ca19d05de8206ff2a4bac0"} Jul 23 09:55:58.700689 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.700601 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4csjr" event={"ID":"3511d554-433a-49ed-87c4-fcf369aa0bf8","Type":"ContainerStarted","Data":"c01a81955b45a1adb14fef2f4ff22fbe2199401a59425deb6bfc382c4659723e"} Jul 23 09:55:58.700689 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.700682 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4csjr" Jul 23 09:55:58.702134 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.702094 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kd464" event={"ID":"c72345fc-45c2-4b0a-9db9-593ae230265e","Type":"ContainerStarted","Data":"abab8d6c39f2081cf0fa132306d2531dd75e093f37971a96d64ccc1105673661"} Jul 23 09:55:58.702247 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.702145 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:55:58.703700 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.703675 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h" event={"ID":"087b3861-7635-4c06-97f9-1226e272dc8e","Type":"ContainerStarted","Data":"bd60de4a2a282cba8165326d69e413ec416669e2350d2cd1518d6ffbdb5e8f6f"} Jul 23 09:55:58.705345 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.705316 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" event={"ID":"811ec163-29b4-44c8-83a2-ee6225cbfbf8","Type":"ContainerStarted","Data":"a0a4b13d68a9cab4b929abe8d9f4fb345bf914ed3d71dd85552ef17d00adc140"} Jul 23 09:55:58.705616 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.705596 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:58.707429 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.707409 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" Jul 23 09:55:58.714591 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.714544 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vw75n" podStartSLOduration=68.449559577 podStartE2EDuration="1m18.714525332s" totalImagesPullingTime="10.264965755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:58.713073486 +0000 UTC m=+78.960029765" watchObservedRunningTime="2026-07-23 09:55:58.714525332 +0000 UTC m=+78.961481605" Jul 23 09:55:58.739267 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.739214 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4csjr" podStartSLOduration=33.587848597 podStartE2EDuration="43.739172232s" totalImagesPullingTime="10.151323635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:58.738831137 +0000 UTC m=+78.985787404" watchObservedRunningTime="2026-07-23 09:55:58.739172232 +0000 UTC m=+78.986128496" Jul 23 09:55:58.752152 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.752091 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kd464" podStartSLOduration=68.506469155 podStartE2EDuration="1m18.752075894s" totalImagesPullingTime="10.245606739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:58.751959868 +0000 UTC m=+78.998916135" watchObservedRunningTime="2026-07-23 09:55:58.752075894 +0000 UTC m=+78.999032160" Jul 23 09:55:58.766622 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.766573 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777455499d-f6d9h" podStartSLOduration=3.288029945 podStartE2EDuration="13.766554738s" totalImagesPullingTime="10.478524793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:55:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:58.765717588 +0000 UTC m=+79.012673856" watchObservedRunningTime="2026-07-23 09:55:58.766554738 +0000 UTC m=+79.013511005" Jul 23 09:55:58.779474 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:58.779405 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cc8d5965d-48c6s" podStartSLOduration=3.536415381 podStartE2EDuration="13.779383022s" totalImagesPullingTime="10.242967641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:55:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:58.779074178 +0000 UTC m=+79.026030444" watchObservedRunningTime="2026-07-23 09:55:58.779383022 +0000 UTC m=+79.026339288" Jul 23 09:55:59.710005 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:59.709961 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8" event={"ID":"9697c23f-7286-4092-86c3-41b8fb851d67","Type":"ContainerStarted","Data":"e0b1b2c44ca98b301824e766f9c4599c717ce15d11ede0b27c14ae2421e6390b"} Jul 23 09:55:59.726274 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:55:59.726219 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8" podStartSLOduration=5.545807512 podStartE2EDuration="6.7261803s" totalImagesPullingTime="1.180372788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:55:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:59.725549684 +0000 UTC m=+79.972505951" watchObservedRunningTime="2026-07-23 09:55:59.7261803 +0000 UTC m=+79.973136565" Jul 23 09:56:00.714427 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:00.714382 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8" Jul 23 09:56:00.719111 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:00.719085 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n7z8" Jul 23 09:56:01.440639 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.440596 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz"] Jul 23 09:56:01.442679 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.442659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.445866 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.445840 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 09:56:01.445866 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.445854 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 23 09:56:01.445866 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.445868 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 09:56:01.446137 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.445854 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 09:56:01.446137 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.445917 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r7knk\"" Jul 23 09:56:01.446137 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.445955 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 23 09:56:01.453598 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.453569 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz"] Jul 23 09:56:01.612484 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.612446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.612484 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.612490 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.612693 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.612508 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.612693 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.612577 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56qw9\" (UniqueName: \"kubernetes.io/projected/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-kube-api-access-56qw9\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.712871 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.712841 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56qw9\" (UniqueName: \"kubernetes.io/projected/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-kube-api-access-56qw9\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.713015 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.712887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.713015 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.712914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.713015 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.712943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.713698 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.713677 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.715638 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.715610 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.715973 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.715643 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.721166 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.721138 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56qw9\" (UniqueName: \"kubernetes.io/projected/2f86cbde-44fe-42d8-a26e-0df389d7fb3e-kube-api-access-56qw9\") pod \"prometheus-operator-5945b5cfd-zt8mz\" (UID: \"2f86cbde-44fe-42d8-a26e-0df389d7fb3e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.752071 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.752031 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" Jul 23 09:56:01.887507 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:01.887472 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz"] Jul 23 09:56:01.890671 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:56:01.890642 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f86cbde_44fe_42d8_a26e_0df389d7fb3e.slice/crio-4338eee5dbf2f9221389bc41644b0dc5a0195210dbab30f5f4512d7c73e8f2ee WatchSource:0}: Error finding container 4338eee5dbf2f9221389bc41644b0dc5a0195210dbab30f5f4512d7c73e8f2ee: Status 404 returned error can't find the container with id 4338eee5dbf2f9221389bc41644b0dc5a0195210dbab30f5f4512d7c73e8f2ee Jul 23 09:56:02.723699 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:02.723658 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" event={"ID":"2f86cbde-44fe-42d8-a26e-0df389d7fb3e","Type":"ContainerStarted","Data":"4338eee5dbf2f9221389bc41644b0dc5a0195210dbab30f5f4512d7c73e8f2ee"} Jul 23 09:56:03.729155 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:03.729123 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" event={"ID":"2f86cbde-44fe-42d8-a26e-0df389d7fb3e","Type":"ContainerStarted","Data":"f9798c6ab7f14b6864141df92688e44a46003c25390466580df02446168a9f90"} Jul 23 09:56:03.729569 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:03.729232 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" event={"ID":"2f86cbde-44fe-42d8-a26e-0df389d7fb3e","Type":"ContainerStarted","Data":"c2208454ad3a3419b825cf840bef2c59c249bc091ae4a94410b09a60ed4c8b4c"} Jul 23 09:56:03.745762 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:03.745707 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zt8mz" podStartSLOduration=1.5586508650000002 podStartE2EDuration="2.745689058s" totalImagesPullingTime="1.187038193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:56:03.744786451 +0000 UTC m=+83.991742731" watchObservedRunningTime="2026-07-23 09:56:03.745689058 +0000 UTC m=+83.992645321" Jul 23 09:56:05.827685 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.827646 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dwjpc"] Jul 23 09:56:05.829908 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.829888 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:05.832275 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.832247 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4ftdv\"" Jul 23 09:56:05.832275 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.832265 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 09:56:05.832275 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.832277 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 09:56:05.832574 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.832326 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 09:56:05.832610 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.832581 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-blfnp"] Jul 23 09:56:05.834960 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.834940 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:05.836845 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.836825 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 23 09:56:05.837002 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.836978 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 23 09:56:05.837002 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.836979 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cfg28\"" Jul 23 09:56:05.837173 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.837107 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 23 09:56:05.853052 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.853018 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-blfnp"] Jul 23 09:56:05.944715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.944677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6cd20837-ec25-45d3-b289-69747da7303d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:05.944715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.944718 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-wtmp\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:05.944930 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.944746 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwjtt\" (UniqueName: \"kubernetes.io/projected/6cd20837-ec25-45d3-b289-69747da7303d-kube-api-access-hwjtt\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:05.944930 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.944769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-accelerators-collector-config\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:05.944930 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.944792 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-tls\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:05.944930 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.944853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-metrics-client-ca\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:05.944930 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.944893 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-sys\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:05.944930 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.944914 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6d2c\" (UniqueName: \"kubernetes.io/projected/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-kube-api-access-f6d2c\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:05.945109 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.944933 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6cd20837-ec25-45d3-b289-69747da7303d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:05.945109 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.944974 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-textfile\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:05.945109 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.945006 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cd20837-ec25-45d3-b289-69747da7303d-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:05.945109 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.945046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:05.945109 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.945067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6cd20837-ec25-45d3-b289-69747da7303d-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:05.945109 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.945085 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-root\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:05.945320 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:05.945114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd20837-ec25-45d3-b289-69747da7303d-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.045668 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.045615 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-wtmp\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.045668 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.045670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwjtt\" (UniqueName: \"kubernetes.io/projected/6cd20837-ec25-45d3-b289-69747da7303d-kube-api-access-hwjtt\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.045882 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.045703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-accelerators-collector-config\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.045882 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.045826 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-wtmp\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.045953 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.045875 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-tls\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.045953 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.045923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-metrics-client-ca\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.046018 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.045952 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-sys\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.046018 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.045980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6d2c\" (UniqueName: \"kubernetes.io/projected/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-kube-api-access-f6d2c\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.046018 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046002 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6cd20837-ec25-45d3-b289-69747da7303d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.046152 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:56:06.046012 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 23 09:56:06.046152 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-textfile\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.046152 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046059 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cd20837-ec25-45d3-b289-69747da7303d-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.046152 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046061 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-sys\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.046152 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:56:06.046078 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-tls podName:c10d452e-f9a5-4eb2-8493-0eefe0a0e848 nodeName:}" failed. No retries permitted until 2026-07-23 09:56:06.546056705 +0000 UTC m=+86.793012948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-tls") pod "node-exporter-dwjpc" (UID: "c10d452e-f9a5-4eb2-8493-0eefe0a0e848") : secret "node-exporter-tls" not found Jul 23 09:56:06.046152 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.046477 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046154 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6cd20837-ec25-45d3-b289-69747da7303d-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.046477 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-root\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.046745 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046713 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd20837-ec25-45d3-b289-69747da7303d-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.046745 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046400 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-accelerators-collector-config\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.046880 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046496 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-metrics-client-ca\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.046880 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046541 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-textfile\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.046880 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046600 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6cd20837-ec25-45d3-b289-69747da7303d-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.046880 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046450 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-root\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.046880 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046804 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6cd20837-ec25-45d3-b289-69747da7303d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.046880 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046854 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cd20837-ec25-45d3-b289-69747da7303d-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.047114 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.046956 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6cd20837-ec25-45d3-b289-69747da7303d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.048714 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.048693 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.049249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.049229 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6cd20837-ec25-45d3-b289-69747da7303d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.049344 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.049309 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd20837-ec25-45d3-b289-69747da7303d-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.066358 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.066324 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6d2c\" (UniqueName: \"kubernetes.io/projected/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-kube-api-access-f6d2c\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.068258 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.068221 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwjtt\" (UniqueName: \"kubernetes.io/projected/6cd20837-ec25-45d3-b289-69747da7303d-kube-api-access-hwjtt\") pod \"kube-state-metrics-56c5594674-blfnp\" (UID: \"6cd20837-ec25-45d3-b289-69747da7303d\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.146800 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.146705 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" Jul 23 09:56:06.300640 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.300605 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-blfnp"] Jul 23 09:56:06.303780 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:56:06.303749 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd20837_ec25_45d3_b289_69747da7303d.slice/crio-f235e424be8b42fca36b7589d222058dcacfcdd498a257a4580a2c4b4675304f WatchSource:0}: Error finding container f235e424be8b42fca36b7589d222058dcacfcdd498a257a4580a2c4b4675304f: Status 404 returned error can't find the container with id f235e424be8b42fca36b7589d222058dcacfcdd498a257a4580a2c4b4675304f Jul 23 09:56:06.551179 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.551139 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-tls\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.553582 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.553562 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c10d452e-f9a5-4eb2-8493-0eefe0a0e848-node-exporter-tls\") pod \"node-exporter-dwjpc\" (UID: \"c10d452e-f9a5-4eb2-8493-0eefe0a0e848\") " pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.740675 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.740649 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dwjpc" Jul 23 09:56:06.741650 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:06.741617 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" event={"ID":"6cd20837-ec25-45d3-b289-69747da7303d","Type":"ContainerStarted","Data":"f235e424be8b42fca36b7589d222058dcacfcdd498a257a4580a2c4b4675304f"} Jul 23 09:56:06.752775 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:56:06.752744 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc10d452e_f9a5_4eb2_8493_0eefe0a0e848.slice/crio-57e89754101f124d318058fd08eaa63bf9e62e520bc8609e04ca04329be5c0c3 WatchSource:0}: Error finding container 57e89754101f124d318058fd08eaa63bf9e62e520bc8609e04ca04329be5c0c3: Status 404 returned error can't find the container with id 57e89754101f124d318058fd08eaa63bf9e62e520bc8609e04ca04329be5c0c3 Jul 23 09:56:07.236277 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:07.236239 2580 patch_prober.go:28] interesting pod/image-registry-59bb6f48dd-6zjbh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 09:56:07.236735 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:07.236304 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" podUID="cece9283-4cb5-45d7-9d63-76d5e19d15b2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 09:56:07.747357 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:07.747306 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwjpc" event={"ID":"c10d452e-f9a5-4eb2-8493-0eefe0a0e848","Type":"ContainerStarted","Data":"57e89754101f124d318058fd08eaa63bf9e62e520bc8609e04ca04329be5c0c3"} Jul 23 09:56:08.713996 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.713963 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4csjr" Jul 23 09:56:08.754500 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.754460 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" event={"ID":"6cd20837-ec25-45d3-b289-69747da7303d","Type":"ContainerStarted","Data":"529ce27b94e77c4418da3ff37cab216b63e9a436dd71f94dc1e51f8a5ceaf21f"} Jul 23 09:56:08.754685 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.754535 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" event={"ID":"6cd20837-ec25-45d3-b289-69747da7303d","Type":"ContainerStarted","Data":"65d0dbea0a5c340ce4fe52cb4ef86407e857705bf1fe9244649012dda7951ed5"} Jul 23 09:56:08.754685 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.754567 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" event={"ID":"6cd20837-ec25-45d3-b289-69747da7303d","Type":"ContainerStarted","Data":"75dc9d133b3ac1c29b4a6a452983556452f467f6a1198df2d8cc147369a092fd"} Jul 23 09:56:08.756343 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.756301 2580 generic.go:356] "Generic (PLEG): container finished" podID="c10d452e-f9a5-4eb2-8493-0eefe0a0e848" containerID="e8b7385c34f8e8e7fdea63883317214f50e7d99f06690892936a42cc68d5bff5" exitCode=0 Jul 23 09:56:08.756469 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.756388 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwjpc" event={"ID":"c10d452e-f9a5-4eb2-8493-0eefe0a0e848","Type":"ContainerDied","Data":"e8b7385c34f8e8e7fdea63883317214f50e7d99f06690892936a42cc68d5bff5"} Jul 23 09:56:08.775773 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.775550 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-blfnp" podStartSLOduration=1.92772175 podStartE2EDuration="3.775529946s" totalImagesPullingTime="1.847808196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:56:08.775312001 +0000 UTC m=+89.022268268" watchObservedRunningTime="2026-07-23 09:56:08.775529946 +0000 UTC m=+89.022486211" Jul 23 09:56:08.902791 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.902753 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-c766dcc55-vxm4v"] Jul 23 09:56:08.905473 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.905446 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:08.907607 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.907580 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 23 09:56:08.907769 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.907652 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ssljg\"" Jul 23 09:56:08.907769 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.907666 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 23 09:56:08.907769 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.907714 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 23 09:56:08.907769 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.907580 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 23 09:56:08.908237 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.908180 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ekt07ecp9hqbo\"" Jul 23 09:56:08.908392 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.908374 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 23 09:56:08.919269 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:08.919232 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-c766dcc55-vxm4v"] Jul 23 09:56:09.056905 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.056814 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59bb6f48dd-6zjbh"] Jul 23 09:56:09.061452 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.061425 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:56:09.072769 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.072731 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.072769 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.072770 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3173cb8c-7f47-48df-a9fd-c3f3004237cd-metrics-client-ca\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.073005 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.072787 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-grpc-tls\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.073005 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.072806 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vfpp\" (UniqueName: \"kubernetes.io/projected/3173cb8c-7f47-48df-a9fd-c3f3004237cd-kube-api-access-8vfpp\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.073005 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.072835 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-tls\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.073005 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.072890 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.073005 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.072918 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.073005 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.072964 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.174148 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.174118 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.174420 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.174158 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.174420 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.174179 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3173cb8c-7f47-48df-a9fd-c3f3004237cd-metrics-client-ca\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.174420 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.174327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-grpc-tls\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.174420 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.174365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vfpp\" (UniqueName: \"kubernetes.io/projected/3173cb8c-7f47-48df-a9fd-c3f3004237cd-kube-api-access-8vfpp\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.174420 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.174394 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-tls\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.174669 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.174440 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.174669 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.174479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.175030 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.175004 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3173cb8c-7f47-48df-a9fd-c3f3004237cd-metrics-client-ca\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.177484 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.177449 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.177574 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.177541 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-grpc-tls\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.177620 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.177596 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.177716 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.177699 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-tls\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.177752 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.177733 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.178106 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.178088 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3173cb8c-7f47-48df-a9fd-c3f3004237cd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.182511 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.182486 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vfpp\" (UniqueName: \"kubernetes.io/projected/3173cb8c-7f47-48df-a9fd-c3f3004237cd-kube-api-access-8vfpp\") pod \"thanos-querier-c766dcc55-vxm4v\" (UID: \"3173cb8c-7f47-48df-a9fd-c3f3004237cd\") " pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.215543 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.215502 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:09.353580 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.353501 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-c766dcc55-vxm4v"] Jul 23 09:56:09.356477 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:56:09.356448 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3173cb8c_7f47_48df_a9fd_c3f3004237cd.slice/crio-477aa07c66b9f13c020b6d6f7bfbccad5d18bc1f32d54cc5b5ec70fea9337f8d WatchSource:0}: Error finding container 477aa07c66b9f13c020b6d6f7bfbccad5d18bc1f32d54cc5b5ec70fea9337f8d: Status 404 returned error can't find the container with id 477aa07c66b9f13c020b6d6f7bfbccad5d18bc1f32d54cc5b5ec70fea9337f8d Jul 23 09:56:09.762802 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.762767 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwjpc" event={"ID":"c10d452e-f9a5-4eb2-8493-0eefe0a0e848","Type":"ContainerStarted","Data":"099d0fa66ea9567f6f6d8eb0d38f403c7f672ceec83f5b46cde0e1766d73cccc"} Jul 23 09:56:09.763395 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.762826 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwjpc" event={"ID":"c10d452e-f9a5-4eb2-8493-0eefe0a0e848","Type":"ContainerStarted","Data":"300a18ed721df75f9c9d0fc5d415ccca7704b29268675836a5059ec061285f57"} Jul 23 09:56:09.763957 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.763933 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" event={"ID":"3173cb8c-7f47-48df-a9fd-c3f3004237cd","Type":"ContainerStarted","Data":"477aa07c66b9f13c020b6d6f7bfbccad5d18bc1f32d54cc5b5ec70fea9337f8d"} Jul 23 09:56:09.780715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:09.780655 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dwjpc" podStartSLOduration=3.271350476 podStartE2EDuration="4.780635985s" totalImagesPullingTime="1.396827186s" totalInitContainerRuntime="112.458323ms" isStatefulPod=true podCreationTimestamp="2026-07-23 09:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:56:09.778890514 +0000 UTC m=+90.025846780" watchObservedRunningTime="2026-07-23 09:56:09.780635985 +0000 UTC m=+90.027592251" Jul 23 09:56:10.048820 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.048732 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-9zqpw"] Jul 23 09:56:10.052022 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.051998 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-9zqpw" Jul 23 09:56:10.055945 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.055881 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 09:56:10.055945 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.055881 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ztcn5\"" Jul 23 09:56:10.056155 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.056062 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 09:56:10.067098 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.067067 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-9zqpw"] Jul 23 09:56:10.182633 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.182599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k67ps\" (UniqueName: \"kubernetes.io/projected/b94460bd-0bb8-4b55-aed9-d59dc00cd0ae-kube-api-access-k67ps\") pod \"downloads-5bc67dcd46-9zqpw\" (UID: \"b94460bd-0bb8-4b55-aed9-d59dc00cd0ae\") " pod="openshift-console/downloads-5bc67dcd46-9zqpw" Jul 23 09:56:10.284308 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.284268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k67ps\" (UniqueName: \"kubernetes.io/projected/b94460bd-0bb8-4b55-aed9-d59dc00cd0ae-kube-api-access-k67ps\") pod \"downloads-5bc67dcd46-9zqpw\" (UID: \"b94460bd-0bb8-4b55-aed9-d59dc00cd0ae\") " pod="openshift-console/downloads-5bc67dcd46-9zqpw" Jul 23 09:56:10.292103 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.292067 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k67ps\" (UniqueName: \"kubernetes.io/projected/b94460bd-0bb8-4b55-aed9-d59dc00cd0ae-kube-api-access-k67ps\") pod \"downloads-5bc67dcd46-9zqpw\" (UID: \"b94460bd-0bb8-4b55-aed9-d59dc00cd0ae\") " pod="openshift-console/downloads-5bc67dcd46-9zqpw" Jul 23 09:56:10.364043 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.363959 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-9zqpw" Jul 23 09:56:10.521589 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.521562 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-9zqpw"] Jul 23 09:56:10.523964 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:56:10.523933 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94460bd_0bb8_4b55_aed9_d59dc00cd0ae.slice/crio-8de6a66ced1fe028e1e8cb1e873325c6ef6bb75a942d00b198e7d35017b62c1d WatchSource:0}: Error finding container 8de6a66ced1fe028e1e8cb1e873325c6ef6bb75a942d00b198e7d35017b62c1d: Status 404 returned error can't find the container with id 8de6a66ced1fe028e1e8cb1e873325c6ef6bb75a942d00b198e7d35017b62c1d Jul 23 09:56:10.769719 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:10.769669 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-9zqpw" event={"ID":"b94460bd-0bb8-4b55-aed9-d59dc00cd0ae","Type":"ContainerStarted","Data":"8de6a66ced1fe028e1e8cb1e873325c6ef6bb75a942d00b198e7d35017b62c1d"} Jul 23 09:56:12.782733 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:12.782694 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" event={"ID":"3173cb8c-7f47-48df-a9fd-c3f3004237cd","Type":"ContainerStarted","Data":"5c8da75df911c34f0d9494f907d7a530599ac492048eeb8ab66ca6610cc337f8"} Jul 23 09:56:12.783358 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:12.782791 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" event={"ID":"3173cb8c-7f47-48df-a9fd-c3f3004237cd","Type":"ContainerStarted","Data":"a4cd04094468a1ec6209083691a60882d65692d299f9632cb59e992a07edc380"} Jul 23 09:56:12.783358 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:12.782832 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" event={"ID":"3173cb8c-7f47-48df-a9fd-c3f3004237cd","Type":"ContainerStarted","Data":"5fb2f021fe1902632a502795513b70a5288fe6f964a0ca95f51569c00b4a9f32"} Jul 23 09:56:13.791224 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:13.791073 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" event={"ID":"3173cb8c-7f47-48df-a9fd-c3f3004237cd","Type":"ContainerStarted","Data":"024743c720fc01a403e6ea75faf4dab7f6455a4b6f7cd07cff82f43880ff73f0"} Jul 23 09:56:13.791224 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:13.791171 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" event={"ID":"3173cb8c-7f47-48df-a9fd-c3f3004237cd","Type":"ContainerStarted","Data":"5cbf58e0c093f9dfabbac76cffbab24a47ec3313f1489db27ee059c922908941"} Jul 23 09:56:13.791761 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:13.791233 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" event={"ID":"3173cb8c-7f47-48df-a9fd-c3f3004237cd","Type":"ContainerStarted","Data":"59f6c4c637eb67236dd137a05ed11e7cade45fea213112496e04f8f0c96d39b7"} Jul 23 09:56:13.791761 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:13.791284 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:13.830213 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:13.830150 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" podStartSLOduration=2.2567475679999998 podStartE2EDuration="5.830131215s" totalImagesPullingTime="3.573383647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:56:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:56:13.827665184 +0000 UTC m=+94.074621461" watchObservedRunningTime="2026-07-23 09:56:13.830131215 +0000 UTC m=+94.077087503" Jul 23 09:56:19.802299 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:19.802271 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-c766dcc55-vxm4v" Jul 23 09:56:27.859260 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:27.859223 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-9zqpw" event={"ID":"b94460bd-0bb8-4b55-aed9-d59dc00cd0ae","Type":"ContainerStarted","Data":"8e3bee3fa77a515df3fe3c43255fd45a13d18f77650da65f64cd6bd3fc3c5ef8"} Jul 23 09:56:27.910655 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:27.910620 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6878979cc4-4g49h"] Jul 23 09:56:27.937522 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:27.937484 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6878979cc4-4g49h"] Jul 23 09:56:27.937676 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:27.937603 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:27.939821 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:27.939795 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 09:56:27.939821 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:27.939811 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 09:56:27.940055 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:27.940035 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 09:56:27.940320 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:27.940302 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 09:56:27.940456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:27.940350 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2wlzg\"" Jul 23 09:56:27.940456 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:27.940405 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 09:56:27.945840 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:27.945781 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 09:56:28.039213 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.039113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngncv\" (UniqueName: \"kubernetes.io/projected/fe682c8e-e92a-4bf6-ba8e-9723ec404053-kube-api-access-ngncv\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.039213 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.039161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-trusted-ca-bundle\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.039404 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.039237 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-oauth-serving-cert\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.039404 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.039264 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-serving-cert\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.039404 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.039314 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-oauth-config\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.039404 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.039344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-config\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.039404 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.039369 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-service-ca\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.140485 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.140443 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-serving-cert\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.140664 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.140499 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-oauth-config\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.140664 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.140538 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-config\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.140664 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.140573 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-service-ca\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.140664 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.140605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngncv\" (UniqueName: \"kubernetes.io/projected/fe682c8e-e92a-4bf6-ba8e-9723ec404053-kube-api-access-ngncv\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.140664 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.140652 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-trusted-ca-bundle\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.140880 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.140703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-oauth-serving-cert\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.141350 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.141323 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-service-ca\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.141483 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.141460 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-config\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.141548 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.141460 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-oauth-serving-cert\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.141951 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.141917 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-trusted-ca-bundle\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.143412 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.143385 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-oauth-config\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.143543 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.143524 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-serving-cert\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.149338 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.149283 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngncv\" (UniqueName: \"kubernetes.io/projected/fe682c8e-e92a-4bf6-ba8e-9723ec404053-kube-api-access-ngncv\") pod \"console-6878979cc4-4g49h\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.248496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.248454 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:28.445374 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.445261 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6878979cc4-4g49h"] Jul 23 09:56:28.452571 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:56:28.452536 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe682c8e_e92a_4bf6_ba8e_9723ec404053.slice/crio-389b7e78e07a6f79fb62f60e98b736436386a4717afe7c6de2e26ec22c2bff99 WatchSource:0}: Error finding container 389b7e78e07a6f79fb62f60e98b736436386a4717afe7c6de2e26ec22c2bff99: Status 404 returned error can't find the container with id 389b7e78e07a6f79fb62f60e98b736436386a4717afe7c6de2e26ec22c2bff99 Jul 23 09:56:28.865870 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.865822 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6878979cc4-4g49h" event={"ID":"fe682c8e-e92a-4bf6-ba8e-9723ec404053","Type":"ContainerStarted","Data":"389b7e78e07a6f79fb62f60e98b736436386a4717afe7c6de2e26ec22c2bff99"} Jul 23 09:56:28.866436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.866412 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-9zqpw" Jul 23 09:56:28.883552 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.883492 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-9zqpw" podStartSLOduration=1.681410884 podStartE2EDuration="18.883469129s" totalImagesPullingTime="17.202058245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:56:28.882935388 +0000 UTC m=+109.129891659" watchObservedRunningTime="2026-07-23 09:56:28.883469129 +0000 UTC m=+109.130425396" Jul 23 09:56:28.883859 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:28.883833 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-9zqpw" Jul 23 09:56:29.714593 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:29.714561 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kd464" Jul 23 09:56:32.887212 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:32.887134 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6878979cc4-4g49h" event={"ID":"fe682c8e-e92a-4bf6-ba8e-9723ec404053","Type":"ContainerStarted","Data":"50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71"} Jul 23 09:56:32.908674 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:32.908600 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6878979cc4-4g49h" podStartSLOduration=2.072058697 podStartE2EDuration="5.908577467s" totalImagesPullingTime="3.83651877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:56:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:56:32.907257828 +0000 UTC m=+113.154214094" watchObservedRunningTime="2026-07-23 09:56:32.908577467 +0000 UTC m=+113.155533737" Jul 23 09:56:34.078262 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.078174 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" podUID="cece9283-4cb5-45d7-9d63-76d5e19d15b2" containerName="registry" containerID="cri-o://58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8" gracePeriod=30 Jul 23 09:56:34.383447 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.383416 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:56:34.505766 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.505719 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-bound-sa-token\") pod \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " Jul 23 09:56:34.505966 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.505777 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-installation-pull-secrets\") pod \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " Jul 23 09:56:34.505966 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.505802 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtq68\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-kube-api-access-gtq68\") pod \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " Jul 23 09:56:34.505966 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.505849 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-trusted-ca\") pod \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " Jul 23 09:56:34.505966 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.505885 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls\") pod \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " Jul 23 09:56:34.505966 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.505928 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-certificates\") pod \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " Jul 23 09:56:34.506249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.505973 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cece9283-4cb5-45d7-9d63-76d5e19d15b2-ca-trust-extracted\") pod \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " Jul 23 09:56:34.506249 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.506010 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-image-registry-private-configuration\") pod \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\" (UID: \"cece9283-4cb5-45d7-9d63-76d5e19d15b2\") " Jul 23 09:56:34.506713 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.506573 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cece9283-4cb5-45d7-9d63-76d5e19d15b2" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:56:34.506826 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.506803 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cece9283-4cb5-45d7-9d63-76d5e19d15b2" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:56:34.510747 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.510707 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cece9283-4cb5-45d7-9d63-76d5e19d15b2" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 09:56:34.512350 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.512319 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cece9283-4cb5-45d7-9d63-76d5e19d15b2" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:56:34.512474 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.512412 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cece9283-4cb5-45d7-9d63-76d5e19d15b2" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:56:34.512624 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.512597 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-kube-api-access-gtq68" (OuterVolumeSpecName: "kube-api-access-gtq68") pod "cece9283-4cb5-45d7-9d63-76d5e19d15b2" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2"). InnerVolumeSpecName "kube-api-access-gtq68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:56:34.513422 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.513397 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cece9283-4cb5-45d7-9d63-76d5e19d15b2" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 09:56:34.517497 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.517469 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cece9283-4cb5-45d7-9d63-76d5e19d15b2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cece9283-4cb5-45d7-9d63-76d5e19d15b2" (UID: "cece9283-4cb5-45d7-9d63-76d5e19d15b2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:56:34.606990 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.606900 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-trusted-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:56:34.606990 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.606937 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-tls\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:56:34.606990 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.606951 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cece9283-4cb5-45d7-9d63-76d5e19d15b2-registry-certificates\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:56:34.606990 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.606968 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cece9283-4cb5-45d7-9d63-76d5e19d15b2-ca-trust-extracted\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:56:34.606990 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.606981 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-image-registry-private-configuration\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:56:34.606990 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.606991 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-bound-sa-token\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:56:34.606990 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.606999 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cece9283-4cb5-45d7-9d63-76d5e19d15b2-installation-pull-secrets\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:56:34.607397 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.607011 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtq68\" (UniqueName: \"kubernetes.io/projected/cece9283-4cb5-45d7-9d63-76d5e19d15b2-kube-api-access-gtq68\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:56:34.898003 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.897917 2580 generic.go:356] "Generic (PLEG): container finished" podID="cece9283-4cb5-45d7-9d63-76d5e19d15b2" containerID="58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8" exitCode=0 Jul 23 09:56:34.898003 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.897975 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" Jul 23 09:56:34.898237 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.897976 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" event={"ID":"cece9283-4cb5-45d7-9d63-76d5e19d15b2","Type":"ContainerDied","Data":"58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8"} Jul 23 09:56:34.898237 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.898112 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59bb6f48dd-6zjbh" event={"ID":"cece9283-4cb5-45d7-9d63-76d5e19d15b2","Type":"ContainerDied","Data":"8ae94208efaf0af9d356b20b328de458699678ed396bf10dabdcbe40993e5e0a"} Jul 23 09:56:34.898237 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.898141 2580 scope.go:117] "RemoveContainer" containerID="58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8" Jul 23 09:56:34.918185 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.917944 2580 scope.go:117] "RemoveContainer" containerID="58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8" Jul 23 09:56:34.918632 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:56:34.918593 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8\": container with ID starting with 58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8 not found: ID does not exist" containerID="58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8" Jul 23 09:56:34.918729 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.918631 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8"} err="failed to get container status \"58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8\": rpc error: code = NotFound desc = could not find container \"58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8\": container with ID starting with 58e6b3f57c7c02f64fb393d145699bec34999d716e8ec5b7e92d5308e2a701b8 not found: ID does not exist" Jul 23 09:56:34.926506 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.926470 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59bb6f48dd-6zjbh"] Jul 23 09:56:34.932239 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:34.932181 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59bb6f48dd-6zjbh"] Jul 23 09:56:36.280496 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:36.280461 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cece9283-4cb5-45d7-9d63-76d5e19d15b2" path="/var/lib/kubelet/pods/cece9283-4cb5-45d7-9d63-76d5e19d15b2/volumes" Jul 23 09:56:38.249603 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:38.249570 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:38.250076 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:38.249625 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:38.254465 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:38.254438 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:56:38.921054 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:56:38.921026 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:57:34.226063 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:34.225980 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6878979cc4-4g49h"] Jul 23 09:57:59.247282 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.247237 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6878979cc4-4g49h" podUID="fe682c8e-e92a-4bf6-ba8e-9723ec404053" containerName="console" containerID="cri-o://50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71" gracePeriod=15 Jul 23 09:57:59.497423 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.497352 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:57:59.557960 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.557922 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-serving-cert\") pod \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " Jul 23 09:57:59.558142 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.557987 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-oauth-serving-cert\") pod \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " Jul 23 09:57:59.558142 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.558018 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-trusted-ca-bundle\") pod \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " Jul 23 09:57:59.558142 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.558070 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-oauth-config\") pod \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " Jul 23 09:57:59.558142 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.558110 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngncv\" (UniqueName: \"kubernetes.io/projected/fe682c8e-e92a-4bf6-ba8e-9723ec404053-kube-api-access-ngncv\") pod \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " Jul 23 09:57:59.558142 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.558133 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-service-ca\") pod \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " Jul 23 09:57:59.558433 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.558165 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-config\") pod \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\" (UID: \"fe682c8e-e92a-4bf6-ba8e-9723ec404053\") " Jul 23 09:57:59.558628 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.558588 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-service-ca" (OuterVolumeSpecName: "service-ca") pod "fe682c8e-e92a-4bf6-ba8e-9723ec404053" (UID: "fe682c8e-e92a-4bf6-ba8e-9723ec404053"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:57:59.558628 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.558611 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fe682c8e-e92a-4bf6-ba8e-9723ec404053" (UID: "fe682c8e-e92a-4bf6-ba8e-9723ec404053"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:57:59.559046 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.558625 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fe682c8e-e92a-4bf6-ba8e-9723ec404053" (UID: "fe682c8e-e92a-4bf6-ba8e-9723ec404053"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:57:59.559046 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.558700 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-config" (OuterVolumeSpecName: "console-config") pod "fe682c8e-e92a-4bf6-ba8e-9723ec404053" (UID: "fe682c8e-e92a-4bf6-ba8e-9723ec404053"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:57:59.560614 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.560580 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe682c8e-e92a-4bf6-ba8e-9723ec404053-kube-api-access-ngncv" (OuterVolumeSpecName: "kube-api-access-ngncv") pod "fe682c8e-e92a-4bf6-ba8e-9723ec404053" (UID: "fe682c8e-e92a-4bf6-ba8e-9723ec404053"). InnerVolumeSpecName "kube-api-access-ngncv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:57:59.560710 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.560619 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fe682c8e-e92a-4bf6-ba8e-9723ec404053" (UID: "fe682c8e-e92a-4bf6-ba8e-9723ec404053"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 09:57:59.560710 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.560628 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fe682c8e-e92a-4bf6-ba8e-9723ec404053" (UID: "fe682c8e-e92a-4bf6-ba8e-9723ec404053"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 09:57:59.659343 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.659304 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngncv\" (UniqueName: \"kubernetes.io/projected/fe682c8e-e92a-4bf6-ba8e-9723ec404053-kube-api-access-ngncv\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:57:59.659343 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.659332 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-service-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:57:59.659343 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.659343 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:57:59.659343 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.659351 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-serving-cert\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:57:59.659581 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.659359 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-oauth-serving-cert\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:57:59.659581 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.659368 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe682c8e-e92a-4bf6-ba8e-9723ec404053-trusted-ca-bundle\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:57:59.659581 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:57:59.659375 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe682c8e-e92a-4bf6-ba8e-9723ec404053-console-oauth-config\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:58:00.210262 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:00.210222 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe682c8e-e92a-4bf6-ba8e-9723ec404053" containerID="50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71" exitCode=0 Jul 23 09:58:00.210458 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:00.210326 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6878979cc4-4g49h" Jul 23 09:58:00.210458 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:00.210329 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6878979cc4-4g49h" event={"ID":"fe682c8e-e92a-4bf6-ba8e-9723ec404053","Type":"ContainerDied","Data":"50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71"} Jul 23 09:58:00.210458 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:00.210378 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6878979cc4-4g49h" event={"ID":"fe682c8e-e92a-4bf6-ba8e-9723ec404053","Type":"ContainerDied","Data":"389b7e78e07a6f79fb62f60e98b736436386a4717afe7c6de2e26ec22c2bff99"} Jul 23 09:58:00.210458 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:00.210399 2580 scope.go:117] "RemoveContainer" containerID="50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71" Jul 23 09:58:00.219697 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:00.219673 2580 scope.go:117] "RemoveContainer" containerID="50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71" Jul 23 09:58:00.220023 ip-10-0-139-55 kubenswrapper[2580]: E0723 09:58:00.220003 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71\": container with ID starting with 50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71 not found: ID does not exist" containerID="50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71" Jul 23 09:58:00.220093 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:00.220033 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71"} err="failed to get container status \"50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71\": rpc error: code = NotFound desc = could not find container \"50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71\": container with ID starting with 50dccf77e2fc24465b31243f13284df08536891fd96eac2c961dfc9542ea7f71 not found: ID does not exist" Jul 23 09:58:00.230787 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:00.230742 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6878979cc4-4g49h"] Jul 23 09:58:00.233786 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:00.233756 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6878979cc4-4g49h"] Jul 23 09:58:00.279001 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:00.278964 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe682c8e-e92a-4bf6-ba8e-9723ec404053" path="/var/lib/kubelet/pods/fe682c8e-e92a-4bf6-ba8e-9723ec404053/volumes" Jul 23 09:58:09.437859 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.437824 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4"] Jul 23 09:58:09.438369 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.438110 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cece9283-4cb5-45d7-9d63-76d5e19d15b2" containerName="registry" Jul 23 09:58:09.438369 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.438121 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cece9283-4cb5-45d7-9d63-76d5e19d15b2" containerName="registry" Jul 23 09:58:09.438369 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.438135 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe682c8e-e92a-4bf6-ba8e-9723ec404053" containerName="console" Jul 23 09:58:09.438369 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.438141 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe682c8e-e92a-4bf6-ba8e-9723ec404053" containerName="console" Jul 23 09:58:09.438811 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.438789 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe682c8e-e92a-4bf6-ba8e-9723ec404053" containerName="console" Jul 23 09:58:09.438872 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.438823 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cece9283-4cb5-45d7-9d63-76d5e19d15b2" containerName="registry" Jul 23 09:58:09.443587 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.443557 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:09.445712 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.445689 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 09:58:09.445964 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.445947 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-l6l4w\"" Jul 23 09:58:09.446417 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.446404 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 09:58:09.449636 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.449606 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4"] Jul 23 09:58:09.535719 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.535682 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cn4l\" (UniqueName: \"kubernetes.io/projected/a8292c8c-f86b-447a-883e-2baf30ad9479-kube-api-access-5cn4l\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:09.535719 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.535726 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:09.535939 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.535753 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:09.637129 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.637089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cn4l\" (UniqueName: \"kubernetes.io/projected/a8292c8c-f86b-447a-883e-2baf30ad9479-kube-api-access-5cn4l\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:09.637129 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.637131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:09.637342 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.637159 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:09.637590 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.637564 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:09.637626 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.637568 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:09.645532 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.645505 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cn4l\" (UniqueName: \"kubernetes.io/projected/a8292c8c-f86b-447a-883e-2baf30ad9479-kube-api-access-5cn4l\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:09.753015 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.752970 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:09.884752 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:09.884728 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4"] Jul 23 09:58:09.887486 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:58:09.887453 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8292c8c_f86b_447a_883e_2baf30ad9479.slice/crio-fa448331bd3f760eeb3e3b2d8cc3a41b13a9361d004ecc0b7090aadffd36b458 WatchSource:0}: Error finding container fa448331bd3f760eeb3e3b2d8cc3a41b13a9361d004ecc0b7090aadffd36b458: Status 404 returned error can't find the container with id fa448331bd3f760eeb3e3b2d8cc3a41b13a9361d004ecc0b7090aadffd36b458 Jul 23 09:58:10.245971 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:10.245934 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" event={"ID":"a8292c8c-f86b-447a-883e-2baf30ad9479","Type":"ContainerStarted","Data":"fa448331bd3f760eeb3e3b2d8cc3a41b13a9361d004ecc0b7090aadffd36b458"} Jul 23 09:58:17.273708 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:17.273667 2580 generic.go:356] "Generic (PLEG): container finished" podID="a8292c8c-f86b-447a-883e-2baf30ad9479" containerID="24beb8ea69562444cbd3f6b623f12860bed7580275b903ee12a3f91be1366b25" exitCode=0 Jul 23 09:58:17.274103 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:17.273721 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" event={"ID":"a8292c8c-f86b-447a-883e-2baf30ad9479","Type":"ContainerDied","Data":"24beb8ea69562444cbd3f6b623f12860bed7580275b903ee12a3f91be1366b25"} Jul 23 09:58:19.283530 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:19.283493 2580 generic.go:356] "Generic (PLEG): container finished" podID="a8292c8c-f86b-447a-883e-2baf30ad9479" containerID="2385d857566bb24868f42b2ce5db1b9949e1a02f00c85e8e6c35b0f2702d61a9" exitCode=0 Jul 23 09:58:19.283917 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:19.283569 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" event={"ID":"a8292c8c-f86b-447a-883e-2baf30ad9479","Type":"ContainerDied","Data":"2385d857566bb24868f42b2ce5db1b9949e1a02f00c85e8e6c35b0f2702d61a9"} Jul 23 09:58:26.313045 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:26.312996 2580 generic.go:356] "Generic (PLEG): container finished" podID="a8292c8c-f86b-447a-883e-2baf30ad9479" containerID="64031074fa0d45c05d84b6eaa225942c68613eada9c892e28a803c65adffab94" exitCode=0 Jul 23 09:58:26.313045 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:26.313029 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" event={"ID":"a8292c8c-f86b-447a-883e-2baf30ad9479","Type":"ContainerDied","Data":"64031074fa0d45c05d84b6eaa225942c68613eada9c892e28a803c65adffab94"} Jul 23 09:58:27.440949 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:27.440926 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:27.592088 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:27.591985 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-util\") pod \"a8292c8c-f86b-447a-883e-2baf30ad9479\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " Jul 23 09:58:27.592088 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:27.592028 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-bundle\") pod \"a8292c8c-f86b-447a-883e-2baf30ad9479\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " Jul 23 09:58:27.592088 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:27.592053 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cn4l\" (UniqueName: \"kubernetes.io/projected/a8292c8c-f86b-447a-883e-2baf30ad9479-kube-api-access-5cn4l\") pod \"a8292c8c-f86b-447a-883e-2baf30ad9479\" (UID: \"a8292c8c-f86b-447a-883e-2baf30ad9479\") " Jul 23 09:58:27.592879 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:27.592850 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-bundle" (OuterVolumeSpecName: "bundle") pod "a8292c8c-f86b-447a-883e-2baf30ad9479" (UID: "a8292c8c-f86b-447a-883e-2baf30ad9479"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:58:27.594448 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:27.594419 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8292c8c-f86b-447a-883e-2baf30ad9479-kube-api-access-5cn4l" (OuterVolumeSpecName: "kube-api-access-5cn4l") pod "a8292c8c-f86b-447a-883e-2baf30ad9479" (UID: "a8292c8c-f86b-447a-883e-2baf30ad9479"). InnerVolumeSpecName "kube-api-access-5cn4l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:58:27.597123 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:27.597098 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-util" (OuterVolumeSpecName: "util") pod "a8292c8c-f86b-447a-883e-2baf30ad9479" (UID: "a8292c8c-f86b-447a-883e-2baf30ad9479"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:58:27.692614 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:27.692581 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-util\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:58:27.692614 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:27.692610 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8292c8c-f86b-447a-883e-2baf30ad9479-bundle\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:58:27.692614 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:27.692619 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cn4l\" (UniqueName: \"kubernetes.io/projected/a8292c8c-f86b-447a-883e-2baf30ad9479-kube-api-access-5cn4l\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:58:28.322052 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:28.322009 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" event={"ID":"a8292c8c-f86b-447a-883e-2baf30ad9479","Type":"ContainerDied","Data":"fa448331bd3f760eeb3e3b2d8cc3a41b13a9361d004ecc0b7090aadffd36b458"} Jul 23 09:58:28.322052 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:28.322032 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dmfx4" Jul 23 09:58:28.322257 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:28.322087 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa448331bd3f760eeb3e3b2d8cc3a41b13a9361d004ecc0b7090aadffd36b458" Jul 23 09:58:32.377291 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.377255 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh"] Jul 23 09:58:32.377771 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.377576 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8292c8c-f86b-447a-883e-2baf30ad9479" containerName="pull" Jul 23 09:58:32.377771 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.377592 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8292c8c-f86b-447a-883e-2baf30ad9479" containerName="pull" Jul 23 09:58:32.377771 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.377608 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8292c8c-f86b-447a-883e-2baf30ad9479" containerName="util" Jul 23 09:58:32.377771 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.377617 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8292c8c-f86b-447a-883e-2baf30ad9479" containerName="util" Jul 23 09:58:32.377771 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.377633 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8292c8c-f86b-447a-883e-2baf30ad9479" containerName="extract" Jul 23 09:58:32.377771 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.377641 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8292c8c-f86b-447a-883e-2baf30ad9479" containerName="extract" Jul 23 09:58:32.377771 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.377703 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8292c8c-f86b-447a-883e-2baf30ad9479" containerName="extract" Jul 23 09:58:32.428421 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.428383 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh"] Jul 23 09:58:32.428615 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.428514 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh" Jul 23 09:58:32.433140 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.433103 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-5zgx6\"" Jul 23 09:58:32.433317 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.433245 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 23 09:58:32.436583 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.436561 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 23 09:58:32.525244 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.525186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2646w\" (UniqueName: \"kubernetes.io/projected/31887716-23f7-40cb-bdf9-c9e311311987-kube-api-access-2646w\") pod \"cert-manager-operator-controller-manager-55c444bf86-flrqh\" (UID: \"31887716-23f7-40cb-bdf9-c9e311311987\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh" Jul 23 09:58:32.525417 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.525263 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31887716-23f7-40cb-bdf9-c9e311311987-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-flrqh\" (UID: \"31887716-23f7-40cb-bdf9-c9e311311987\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh" Jul 23 09:58:32.626397 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.626357 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31887716-23f7-40cb-bdf9-c9e311311987-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-flrqh\" (UID: \"31887716-23f7-40cb-bdf9-c9e311311987\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh" Jul 23 09:58:32.626511 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.626458 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2646w\" (UniqueName: \"kubernetes.io/projected/31887716-23f7-40cb-bdf9-c9e311311987-kube-api-access-2646w\") pod \"cert-manager-operator-controller-manager-55c444bf86-flrqh\" (UID: \"31887716-23f7-40cb-bdf9-c9e311311987\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh" Jul 23 09:58:32.626823 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.626798 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31887716-23f7-40cb-bdf9-c9e311311987-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-flrqh\" (UID: \"31887716-23f7-40cb-bdf9-c9e311311987\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh" Jul 23 09:58:32.634094 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.634018 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2646w\" (UniqueName: \"kubernetes.io/projected/31887716-23f7-40cb-bdf9-c9e311311987-kube-api-access-2646w\") pod \"cert-manager-operator-controller-manager-55c444bf86-flrqh\" (UID: \"31887716-23f7-40cb-bdf9-c9e311311987\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh" Jul 23 09:58:32.737114 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.737080 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh" Jul 23 09:58:32.891515 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:32.891420 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh"] Jul 23 09:58:32.895363 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:58:32.895334 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31887716_23f7_40cb_bdf9_c9e311311987.slice/crio-96eecae319f7d782c4fe32fb312d0aeccfd4d45ef9439375e103d447bc31f834 WatchSource:0}: Error finding container 96eecae319f7d782c4fe32fb312d0aeccfd4d45ef9439375e103d447bc31f834: Status 404 returned error can't find the container with id 96eecae319f7d782c4fe32fb312d0aeccfd4d45ef9439375e103d447bc31f834 Jul 23 09:58:33.343597 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:33.343562 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh" event={"ID":"31887716-23f7-40cb-bdf9-c9e311311987","Type":"ContainerStarted","Data":"96eecae319f7d782c4fe32fb312d0aeccfd4d45ef9439375e103d447bc31f834"} Jul 23 09:58:35.353695 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:35.353612 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh" event={"ID":"31887716-23f7-40cb-bdf9-c9e311311987","Type":"ContainerStarted","Data":"9fb6a8c6aea0bb3493976b12f6333c81af2a739aa226a84c91e3cca936083029"} Jul 23 09:58:35.374126 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:35.374068 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-flrqh" podStartSLOduration=1.208649415 podStartE2EDuration="3.374046641s" totalImagesPullingTime="2.165397226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:58:35.372148351 +0000 UTC m=+235.619104620" watchObservedRunningTime="2026-07-23 09:58:35.374046641 +0000 UTC m=+235.621002908" Jul 23 09:58:36.279657 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.279618 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn"] Jul 23 09:58:36.301085 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.301053 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn"] Jul 23 09:58:36.301275 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.301179 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:36.303538 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.303504 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 09:58:36.303986 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.303970 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 09:58:36.304050 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.303998 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-l6l4w\"" Jul 23 09:58:36.356548 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.356517 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:36.356923 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.356590 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:36.356923 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.356615 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlmdd\" (UniqueName: \"kubernetes.io/projected/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-kube-api-access-vlmdd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:36.456888 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.456852 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:36.456888 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.456886 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlmdd\" (UniqueName: \"kubernetes.io/projected/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-kube-api-access-vlmdd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:36.457118 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.457094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:36.457367 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.457347 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:36.457430 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.457411 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:36.464595 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.464562 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlmdd\" (UniqueName: \"kubernetes.io/projected/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-kube-api-access-vlmdd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:36.610428 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.610351 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:36.758174 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:36.758140 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn"] Jul 23 09:58:36.761225 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:58:36.761169 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6061e0ff_a08a_46a4_bcb5_0bf88cd4301d.slice/crio-a7b195bbd169ab8caec9a7ec7049dea9132054ddbcfeda64bc30b38ab4380652 WatchSource:0}: Error finding container a7b195bbd169ab8caec9a7ec7049dea9132054ddbcfeda64bc30b38ab4380652: Status 404 returned error can't find the container with id a7b195bbd169ab8caec9a7ec7049dea9132054ddbcfeda64bc30b38ab4380652 Jul 23 09:58:37.363945 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:37.363908 2580 generic.go:356] "Generic (PLEG): container finished" podID="6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" containerID="c04a0ab22c2f6c912164ffe651c7000e79b430c9fe836347f730b94126b726e3" exitCode=0 Jul 23 09:58:37.364440 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:37.363988 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" event={"ID":"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d","Type":"ContainerDied","Data":"c04a0ab22c2f6c912164ffe651c7000e79b430c9fe836347f730b94126b726e3"} Jul 23 09:58:37.364440 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:37.364037 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" event={"ID":"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d","Type":"ContainerStarted","Data":"a7b195bbd169ab8caec9a7ec7049dea9132054ddbcfeda64bc30b38ab4380652"} Jul 23 09:58:39.679406 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.679373 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz"] Jul 23 09:58:39.681371 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.681349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz" Jul 23 09:58:39.683864 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.683848 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-pb25q\"" Jul 23 09:58:39.689504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.689479 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 23 09:58:39.689504 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.689499 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 23 09:58:39.705436 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.705411 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz"] Jul 23 09:58:39.783607 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.783571 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsdq8\" (UniqueName: \"kubernetes.io/projected/e9c81887-f84f-4a0e-9e39-13c0f23c550b-kube-api-access-zsdq8\") pod \"cert-manager-cainjector-559cdd8c87-5ksvz\" (UID: \"e9c81887-f84f-4a0e-9e39-13c0f23c550b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz" Jul 23 09:58:39.783770 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.783627 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9c81887-f84f-4a0e-9e39-13c0f23c550b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5ksvz\" (UID: \"e9c81887-f84f-4a0e-9e39-13c0f23c550b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz" Jul 23 09:58:39.884224 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.884169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsdq8\" (UniqueName: \"kubernetes.io/projected/e9c81887-f84f-4a0e-9e39-13c0f23c550b-kube-api-access-zsdq8\") pod \"cert-manager-cainjector-559cdd8c87-5ksvz\" (UID: \"e9c81887-f84f-4a0e-9e39-13c0f23c550b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz" Jul 23 09:58:39.884400 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.884251 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9c81887-f84f-4a0e-9e39-13c0f23c550b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5ksvz\" (UID: \"e9c81887-f84f-4a0e-9e39-13c0f23c550b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz" Jul 23 09:58:39.892113 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.892083 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsdq8\" (UniqueName: \"kubernetes.io/projected/e9c81887-f84f-4a0e-9e39-13c0f23c550b-kube-api-access-zsdq8\") pod \"cert-manager-cainjector-559cdd8c87-5ksvz\" (UID: \"e9c81887-f84f-4a0e-9e39-13c0f23c550b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz" Jul 23 09:58:39.892113 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.892096 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9c81887-f84f-4a0e-9e39-13c0f23c550b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5ksvz\" (UID: \"e9c81887-f84f-4a0e-9e39-13c0f23c550b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz" Jul 23 09:58:39.990827 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:39.990796 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz" Jul 23 09:58:40.117711 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:40.117677 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz"] Jul 23 09:58:40.120362 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:58:40.120328 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9c81887_f84f_4a0e_9e39_13c0f23c550b.slice/crio-291906b8fece60f1a412c8a8bdf0dc4ec02ea57b4fabc74a1b4f8f74f72f20e6 WatchSource:0}: Error finding container 291906b8fece60f1a412c8a8bdf0dc4ec02ea57b4fabc74a1b4f8f74f72f20e6: Status 404 returned error can't find the container with id 291906b8fece60f1a412c8a8bdf0dc4ec02ea57b4fabc74a1b4f8f74f72f20e6 Jul 23 09:58:40.378023 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:40.377918 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz" event={"ID":"e9c81887-f84f-4a0e-9e39-13c0f23c550b","Type":"ContainerStarted","Data":"291906b8fece60f1a412c8a8bdf0dc4ec02ea57b4fabc74a1b4f8f74f72f20e6"} Jul 23 09:58:40.379591 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:40.379564 2580 generic.go:356] "Generic (PLEG): container finished" podID="6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" containerID="7b90b58d9768efbfafee28bb6285807ebe49da1ef78ed9c5208f29aa39549bf3" exitCode=0 Jul 23 09:58:40.379754 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:40.379621 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" event={"ID":"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d","Type":"ContainerDied","Data":"7b90b58d9768efbfafee28bb6285807ebe49da1ef78ed9c5208f29aa39549bf3"} Jul 23 09:58:41.386621 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:41.386577 2580 generic.go:356] "Generic (PLEG): container finished" podID="6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" containerID="d728d9897f67e05c82ad0bcd03366b247c59ca866c15ef23f3275411b717024a" exitCode=0 Jul 23 09:58:41.387396 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:41.386669 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" event={"ID":"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d","Type":"ContainerDied","Data":"d728d9897f67e05c82ad0bcd03366b247c59ca866c15ef23f3275411b717024a"} Jul 23 09:58:42.609534 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:42.609508 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:42.705646 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:42.705609 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-bundle\") pod \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " Jul 23 09:58:42.705820 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:42.705668 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlmdd\" (UniqueName: \"kubernetes.io/projected/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-kube-api-access-vlmdd\") pod \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " Jul 23 09:58:42.705820 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:42.705688 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-util\") pod \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\" (UID: \"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d\") " Jul 23 09:58:42.706038 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:42.706012 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-bundle" (OuterVolumeSpecName: "bundle") pod "6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" (UID: "6061e0ff-a08a-46a4-bcb5-0bf88cd4301d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:58:42.708026 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:42.707995 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-kube-api-access-vlmdd" (OuterVolumeSpecName: "kube-api-access-vlmdd") pod "6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" (UID: "6061e0ff-a08a-46a4-bcb5-0bf88cd4301d"). InnerVolumeSpecName "kube-api-access-vlmdd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:58:42.712572 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:42.712541 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-util" (OuterVolumeSpecName: "util") pod "6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" (UID: "6061e0ff-a08a-46a4-bcb5-0bf88cd4301d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:58:42.806546 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:42.806507 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-bundle\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:58:42.806546 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:42.806538 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlmdd\" (UniqueName: \"kubernetes.io/projected/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-kube-api-access-vlmdd\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:58:42.806546 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:42.806547 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6061e0ff-a08a-46a4-bcb5-0bf88cd4301d-util\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:58:43.398208 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:43.398146 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz" event={"ID":"e9c81887-f84f-4a0e-9e39-13c0f23c550b","Type":"ContainerStarted","Data":"b8ab4d9798db6b809b65ef0e2ebdb215b5c0f8303fd794309238642e0968616c"} Jul 23 09:58:43.400092 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:43.400068 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" event={"ID":"6061e0ff-a08a-46a4-bcb5-0bf88cd4301d","Type":"ContainerDied","Data":"a7b195bbd169ab8caec9a7ec7049dea9132054ddbcfeda64bc30b38ab4380652"} Jul 23 09:58:43.400244 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:43.400107 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7b195bbd169ab8caec9a7ec7049dea9132054ddbcfeda64bc30b38ab4380652" Jul 23 09:58:43.400244 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:43.400114 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmrgn" Jul 23 09:58:43.412564 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:43.412517 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5ksvz" podStartSLOduration=1.9073013300000001 podStartE2EDuration="4.412502056s" totalImagesPullingTime="2.505200726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:58:43.411762727 +0000 UTC m=+243.658718994" watchObservedRunningTime="2026-07-23 09:58:43.412502056 +0000 UTC m=+243.659458321" Jul 23 09:58:50.228241 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.228209 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh"] Jul 23 09:58:50.228634 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.228495 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" containerName="pull" Jul 23 09:58:50.228634 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.228506 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" containerName="pull" Jul 23 09:58:50.228634 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.228522 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" containerName="util" Jul 23 09:58:50.228634 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.228527 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" containerName="util" Jul 23 09:58:50.228634 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.228533 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" containerName="extract" Jul 23 09:58:50.228634 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.228538 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" containerName="extract" Jul 23 09:58:50.228634 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.228578 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6061e0ff-a08a-46a4-bcb5-0bf88cd4301d" containerName="extract" Jul 23 09:58:50.230492 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.230477 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh" Jul 23 09:58:50.232570 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.232542 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 09:58:50.232725 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.232542 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 09:58:50.232949 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.232934 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-pzprx\"" Jul 23 09:58:50.238964 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.238943 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh"] Jul 23 09:58:50.262610 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.262581 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a53d146-b071-4217-ad9d-a7bc9f1cf7e3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sj7kh\" (UID: \"5a53d146-b071-4217-ad9d-a7bc9f1cf7e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh" Jul 23 09:58:50.262810 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.262666 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m24t2\" (UniqueName: \"kubernetes.io/projected/5a53d146-b071-4217-ad9d-a7bc9f1cf7e3-kube-api-access-m24t2\") pod \"openshift-lws-operator-fd8ccff4c-sj7kh\" (UID: \"5a53d146-b071-4217-ad9d-a7bc9f1cf7e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh" Jul 23 09:58:50.363666 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.363628 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a53d146-b071-4217-ad9d-a7bc9f1cf7e3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sj7kh\" (UID: \"5a53d146-b071-4217-ad9d-a7bc9f1cf7e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh" Jul 23 09:58:50.363861 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.363729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m24t2\" (UniqueName: \"kubernetes.io/projected/5a53d146-b071-4217-ad9d-a7bc9f1cf7e3-kube-api-access-m24t2\") pod \"openshift-lws-operator-fd8ccff4c-sj7kh\" (UID: \"5a53d146-b071-4217-ad9d-a7bc9f1cf7e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh" Jul 23 09:58:50.364145 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.364117 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a53d146-b071-4217-ad9d-a7bc9f1cf7e3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sj7kh\" (UID: \"5a53d146-b071-4217-ad9d-a7bc9f1cf7e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh" Jul 23 09:58:50.372746 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.372704 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m24t2\" (UniqueName: \"kubernetes.io/projected/5a53d146-b071-4217-ad9d-a7bc9f1cf7e3-kube-api-access-m24t2\") pod \"openshift-lws-operator-fd8ccff4c-sj7kh\" (UID: \"5a53d146-b071-4217-ad9d-a7bc9f1cf7e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh" Jul 23 09:58:50.540313 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.540220 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh" Jul 23 09:58:50.670435 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:50.670386 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh"] Jul 23 09:58:50.673160 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:58:50.673133 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a53d146_b071_4217_ad9d_a7bc9f1cf7e3.slice/crio-3ac477ac260a1ea57cfa57214488ba4d69d45372a5e4b37334b6cda27886cf76 WatchSource:0}: Error finding container 3ac477ac260a1ea57cfa57214488ba4d69d45372a5e4b37334b6cda27886cf76: Status 404 returned error can't find the container with id 3ac477ac260a1ea57cfa57214488ba4d69d45372a5e4b37334b6cda27886cf76 Jul 23 09:58:51.434846 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:51.434811 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh" event={"ID":"5a53d146-b071-4217-ad9d-a7bc9f1cf7e3","Type":"ContainerStarted","Data":"3ac477ac260a1ea57cfa57214488ba4d69d45372a5e4b37334b6cda27886cf76"} Jul 23 09:58:52.440660 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:52.440629 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh" event={"ID":"5a53d146-b071-4217-ad9d-a7bc9f1cf7e3","Type":"ContainerStarted","Data":"366ca3f9ae65a708669e1042e75ea2ea8970bcd68ca062cc9a49e280e74e49b0"} Jul 23 09:58:52.456314 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:52.456250 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sj7kh" podStartSLOduration=0.932510278 podStartE2EDuration="2.456224811s" totalImagesPullingTime="1.523714533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:58:52.454761172 +0000 UTC m=+252.701717435" watchObservedRunningTime="2026-07-23 09:58:52.456224811 +0000 UTC m=+252.703181076" Jul 23 09:58:55.333273 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.333235 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s"] Jul 23 09:58:55.335813 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.335796 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:55.337862 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.337837 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 09:58:55.337862 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.337860 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 09:58:55.338459 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.338442 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-l6l4w\"" Jul 23 09:58:55.344717 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.344692 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s"] Jul 23 09:58:55.405849 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.405809 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hktz\" (UniqueName: \"kubernetes.io/projected/f79a1897-cccc-4950-b0c0-8910df82ad3a-kube-api-access-2hktz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:55.405849 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.405854 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:55.406085 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.405935 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:55.507164 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.507127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:55.507405 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.507217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hktz\" (UniqueName: \"kubernetes.io/projected/f79a1897-cccc-4950-b0c0-8910df82ad3a-kube-api-access-2hktz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:55.507405 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.507259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:55.507642 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.507618 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:55.507691 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.507630 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:55.515696 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.515666 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hktz\" (UniqueName: \"kubernetes.io/projected/f79a1897-cccc-4950-b0c0-8910df82ad3a-kube-api-access-2hktz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:55.645316 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.645222 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:55.785743 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:55.785717 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s"] Jul 23 09:58:55.788219 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:58:55.788175 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79a1897_cccc_4950_b0c0_8910df82ad3a.slice/crio-7d403ae739fb76666e4ddbd6f98d61add2a33e4945ad7741d96fa5c2b5fd681e WatchSource:0}: Error finding container 7d403ae739fb76666e4ddbd6f98d61add2a33e4945ad7741d96fa5c2b5fd681e: Status 404 returned error can't find the container with id 7d403ae739fb76666e4ddbd6f98d61add2a33e4945ad7741d96fa5c2b5fd681e Jul 23 09:58:56.458845 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:56.458808 2580 generic.go:356] "Generic (PLEG): container finished" podID="f79a1897-cccc-4950-b0c0-8910df82ad3a" containerID="109ef73a90bc3c43bbdbdd7c1649230c948c6bed7b6e8e56a94391a67c088926" exitCode=0 Jul 23 09:58:56.459296 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:56.458909 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" event={"ID":"f79a1897-cccc-4950-b0c0-8910df82ad3a","Type":"ContainerDied","Data":"109ef73a90bc3c43bbdbdd7c1649230c948c6bed7b6e8e56a94391a67c088926"} Jul 23 09:58:56.459296 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:56.458971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" event={"ID":"f79a1897-cccc-4950-b0c0-8910df82ad3a","Type":"ContainerStarted","Data":"7d403ae739fb76666e4ddbd6f98d61add2a33e4945ad7741d96fa5c2b5fd681e"} Jul 23 09:58:57.464528 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:57.464492 2580 generic.go:356] "Generic (PLEG): container finished" podID="f79a1897-cccc-4950-b0c0-8910df82ad3a" containerID="13a42d7dacdc48db0d258272a8f43a7d9b1cd82352390a5e063a0c78c1026bfa" exitCode=0 Jul 23 09:58:57.464914 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:57.464526 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" event={"ID":"f79a1897-cccc-4950-b0c0-8910df82ad3a","Type":"ContainerDied","Data":"13a42d7dacdc48db0d258272a8f43a7d9b1cd82352390a5e063a0c78c1026bfa"} Jul 23 09:58:58.471326 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:58.471295 2580 generic.go:356] "Generic (PLEG): container finished" podID="f79a1897-cccc-4950-b0c0-8910df82ad3a" containerID="126c18ae3699d31c79e38e4f8db8468b8444bd88be636d667ad85d8fec45fa98" exitCode=0 Jul 23 09:58:58.471715 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:58.471367 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" event={"ID":"f79a1897-cccc-4950-b0c0-8910df82ad3a","Type":"ContainerDied","Data":"126c18ae3699d31c79e38e4f8db8468b8444bd88be636d667ad85d8fec45fa98"} Jul 23 09:58:59.600011 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:59.599984 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:58:59.647014 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:59.646980 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-bundle\") pod \"f79a1897-cccc-4950-b0c0-8910df82ad3a\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " Jul 23 09:58:59.647242 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:59.647031 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hktz\" (UniqueName: \"kubernetes.io/projected/f79a1897-cccc-4950-b0c0-8910df82ad3a-kube-api-access-2hktz\") pod \"f79a1897-cccc-4950-b0c0-8910df82ad3a\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " Jul 23 09:58:59.647242 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:59.647062 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-util\") pod \"f79a1897-cccc-4950-b0c0-8910df82ad3a\" (UID: \"f79a1897-cccc-4950-b0c0-8910df82ad3a\") " Jul 23 09:58:59.647822 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:59.647793 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-bundle" (OuterVolumeSpecName: "bundle") pod "f79a1897-cccc-4950-b0c0-8910df82ad3a" (UID: "f79a1897-cccc-4950-b0c0-8910df82ad3a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:58:59.649328 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:59.649295 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f79a1897-cccc-4950-b0c0-8910df82ad3a-kube-api-access-2hktz" (OuterVolumeSpecName: "kube-api-access-2hktz") pod "f79a1897-cccc-4950-b0c0-8910df82ad3a" (UID: "f79a1897-cccc-4950-b0c0-8910df82ad3a"). InnerVolumeSpecName "kube-api-access-2hktz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:58:59.652543 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:59.652507 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-util" (OuterVolumeSpecName: "util") pod "f79a1897-cccc-4950-b0c0-8910df82ad3a" (UID: "f79a1897-cccc-4950-b0c0-8910df82ad3a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:58:59.748669 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:59.748580 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-bundle\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:58:59.748669 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:59.748610 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hktz\" (UniqueName: \"kubernetes.io/projected/f79a1897-cccc-4950-b0c0-8910df82ad3a-kube-api-access-2hktz\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:58:59.748669 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:58:59.748621 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f79a1897-cccc-4950-b0c0-8910df82ad3a-util\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 09:59:00.481420 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:00.481386 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" event={"ID":"f79a1897-cccc-4950-b0c0-8910df82ad3a","Type":"ContainerDied","Data":"7d403ae739fb76666e4ddbd6f98d61add2a33e4945ad7741d96fa5c2b5fd681e"} Jul 23 09:59:00.481420 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:00.481409 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affdd5s" Jul 23 09:59:00.481622 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:00.481432 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d403ae739fb76666e4ddbd6f98d61add2a33e4945ad7741d96fa5c2b5fd681e" Jul 23 09:59:10.407557 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.407469 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9"] Jul 23 09:59:10.408015 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.407756 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79a1897-cccc-4950-b0c0-8910df82ad3a" containerName="extract" Jul 23 09:59:10.408015 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.407766 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79a1897-cccc-4950-b0c0-8910df82ad3a" containerName="extract" Jul 23 09:59:10.408015 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.407780 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79a1897-cccc-4950-b0c0-8910df82ad3a" containerName="pull" Jul 23 09:59:10.408015 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.407786 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79a1897-cccc-4950-b0c0-8910df82ad3a" containerName="pull" Jul 23 09:59:10.408015 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.407796 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79a1897-cccc-4950-b0c0-8910df82ad3a" containerName="util" Jul 23 09:59:10.408015 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.407802 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79a1897-cccc-4950-b0c0-8910df82ad3a" containerName="util" Jul 23 09:59:10.408015 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.407853 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f79a1897-cccc-4950-b0c0-8910df82ad3a" containerName="extract" Jul 23 09:59:10.415413 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.415384 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.417358 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.417334 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 09:59:10.417540 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.417448 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 09:59:10.417949 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.417822 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 09:59:10.417949 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.417823 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5pgqj\"" Jul 23 09:59:10.417949 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.417914 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 09:59:10.425539 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.425509 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9"] Jul 23 09:59:10.534682 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.534647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43617c9f-db83-4f72-ac31-4320dc43aa7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.534682 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.534683 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/43617c9f-db83-4f72-ac31-4320dc43aa7f-manifests\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.534907 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.534709 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d67l9\" (UniqueName: \"kubernetes.io/projected/43617c9f-db83-4f72-ac31-4320dc43aa7f-kube-api-access-d67l9\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.534907 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.534799 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43617c9f-db83-4f72-ac31-4320dc43aa7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.534907 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.534832 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43617c9f-db83-4f72-ac31-4320dc43aa7f-tmp\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.635451 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.635409 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43617c9f-db83-4f72-ac31-4320dc43aa7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.635451 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.635450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/43617c9f-db83-4f72-ac31-4320dc43aa7f-manifests\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.635678 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.635478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d67l9\" (UniqueName: \"kubernetes.io/projected/43617c9f-db83-4f72-ac31-4320dc43aa7f-kube-api-access-d67l9\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.635678 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.635522 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43617c9f-db83-4f72-ac31-4320dc43aa7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.635678 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.635582 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43617c9f-db83-4f72-ac31-4320dc43aa7f-tmp\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.635899 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.635875 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/43617c9f-db83-4f72-ac31-4320dc43aa7f-manifests\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.638115 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.638082 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43617c9f-db83-4f72-ac31-4320dc43aa7f-tmp\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.638279 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.638171 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43617c9f-db83-4f72-ac31-4320dc43aa7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.638279 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.638213 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43617c9f-db83-4f72-ac31-4320dc43aa7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.643229 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.643184 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d67l9\" (UniqueName: \"kubernetes.io/projected/43617c9f-db83-4f72-ac31-4320dc43aa7f-kube-api-access-d67l9\") pod \"opendatahub-operator-controller-manager-86bc8b458c-2zwn9\" (UID: \"43617c9f-db83-4f72-ac31-4320dc43aa7f\") " pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.727566 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.727531 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:10.877746 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:10.877720 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9"] Jul 23 09:59:10.879780 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:59:10.879751 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43617c9f_db83_4f72_ac31_4320dc43aa7f.slice/crio-e1dd13632297c4dd03aca13ec823b40954b8527475ce6b48d494bba9730039ac WatchSource:0}: Error finding container e1dd13632297c4dd03aca13ec823b40954b8527475ce6b48d494bba9730039ac: Status 404 returned error can't find the container with id e1dd13632297c4dd03aca13ec823b40954b8527475ce6b48d494bba9730039ac Jul 23 09:59:11.531431 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:11.531376 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" event={"ID":"43617c9f-db83-4f72-ac31-4320dc43aa7f","Type":"ContainerStarted","Data":"e1dd13632297c4dd03aca13ec823b40954b8527475ce6b48d494bba9730039ac"} Jul 23 09:59:14.547287 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:14.547245 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" event={"ID":"43617c9f-db83-4f72-ac31-4320dc43aa7f","Type":"ContainerStarted","Data":"041d13c04337676648f9213f2dc9cf119895b18809ef123cdfb65b3403db999e"} Jul 23 09:59:15.553975 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:15.553939 2580 generic.go:356] "Generic (PLEG): container finished" podID="43617c9f-db83-4f72-ac31-4320dc43aa7f" containerID="041d13c04337676648f9213f2dc9cf119895b18809ef123cdfb65b3403db999e" exitCode=0 Jul 23 09:59:15.553975 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:15.553980 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" event={"ID":"43617c9f-db83-4f72-ac31-4320dc43aa7f","Type":"ContainerDied","Data":"041d13c04337676648f9213f2dc9cf119895b18809ef123cdfb65b3403db999e"} Jul 23 09:59:16.560737 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:16.560693 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" event={"ID":"43617c9f-db83-4f72-ac31-4320dc43aa7f","Type":"ContainerStarted","Data":"33829d4956c29271651a3f81cc131f72429017c9b59652633003d3b0f533c06f"} Jul 23 09:59:16.561239 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:16.560890 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:16.580990 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:16.580937 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" podStartSLOduration=2.01308758 podStartE2EDuration="6.580918765s" totalImagesPullingTime="3.533152832s" totalInitContainerRuntime="1.034678353s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:59:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:59:16.579419232 +0000 UTC m=+276.826375503" watchObservedRunningTime="2026-07-23 09:59:16.580918765 +0000 UTC m=+276.827875030" Jul 23 09:59:27.568348 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.568314 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86bc8b458c-2zwn9" Jul 23 09:59:27.746919 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.746882 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5"] Jul 23 09:59:27.757285 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.757254 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.757710 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.757683 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5"] Jul 23 09:59:27.759257 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.759233 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 23 09:59:27.759395 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.759236 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 23 09:59:27.759395 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.759338 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-r6wxx\"" Jul 23 09:59:27.759395 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.759376 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 23 09:59:27.877691 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.877590 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2a5d3298-81cb-40f8-8dab-62d57711652c-metrics-cert\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.877691 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.877634 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmwbt\" (UniqueName: \"kubernetes.io/projected/2a5d3298-81cb-40f8-8dab-62d57711652c-kube-api-access-dmwbt\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.877691 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.877661 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2a5d3298-81cb-40f8-8dab-62d57711652c-manager-config\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.877920 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.877719 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a5d3298-81cb-40f8-8dab-62d57711652c-cert\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.979041 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.979002 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2a5d3298-81cb-40f8-8dab-62d57711652c-metrics-cert\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.979041 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.979044 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmwbt\" (UniqueName: \"kubernetes.io/projected/2a5d3298-81cb-40f8-8dab-62d57711652c-kube-api-access-dmwbt\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.979403 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.979079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2a5d3298-81cb-40f8-8dab-62d57711652c-manager-config\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.979403 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.979107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a5d3298-81cb-40f8-8dab-62d57711652c-cert\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.979810 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.979787 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2a5d3298-81cb-40f8-8dab-62d57711652c-manager-config\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.981784 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.981757 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2a5d3298-81cb-40f8-8dab-62d57711652c-metrics-cert\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.981886 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.981805 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a5d3298-81cb-40f8-8dab-62d57711652c-cert\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:27.989717 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:27.989691 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmwbt\" (UniqueName: \"kubernetes.io/projected/2a5d3298-81cb-40f8-8dab-62d57711652c-kube-api-access-dmwbt\") pod \"lws-controller-manager-6d6fb6dcb8-8jmw5\" (UID: \"2a5d3298-81cb-40f8-8dab-62d57711652c\") " pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:28.067758 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:28.067714 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:28.207049 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:28.207012 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5"] Jul 23 09:59:28.210955 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:59:28.210931 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a5d3298_81cb_40f8_8dab_62d57711652c.slice/crio-9a1bc300d046e396a5e412ca0752d8e3288035fa11adf903b15ded4e91ea2f7f WatchSource:0}: Error finding container 9a1bc300d046e396a5e412ca0752d8e3288035fa11adf903b15ded4e91ea2f7f: Status 404 returned error can't find the container with id 9a1bc300d046e396a5e412ca0752d8e3288035fa11adf903b15ded4e91ea2f7f Jul 23 09:59:28.615716 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:28.615681 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" event={"ID":"2a5d3298-81cb-40f8-8dab-62d57711652c","Type":"ContainerStarted","Data":"9a1bc300d046e396a5e412ca0752d8e3288035fa11adf903b15ded4e91ea2f7f"} Jul 23 09:59:30.029268 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.029233 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-565557bd5f-xpngg"] Jul 23 09:59:30.032783 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.032762 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" Jul 23 09:59:30.035427 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.035388 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 09:59:30.035427 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.035410 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 09:59:30.035615 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.035458 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 09:59:30.035948 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.035933 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sf9r5\"" Jul 23 09:59:30.038833 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.038661 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 09:59:30.059983 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.059891 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-565557bd5f-xpngg"] Jul 23 09:59:30.194501 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.194458 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21248380-e684-4a60-b641-fc605527e60c-tls-certs\") pod \"kube-auth-proxy-565557bd5f-xpngg\" (UID: \"21248380-e684-4a60-b641-fc605527e60c\") " pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" Jul 23 09:59:30.194652 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.194509 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4s7z\" (UniqueName: \"kubernetes.io/projected/21248380-e684-4a60-b641-fc605527e60c-kube-api-access-x4s7z\") pod \"kube-auth-proxy-565557bd5f-xpngg\" (UID: \"21248380-e684-4a60-b641-fc605527e60c\") " pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" Jul 23 09:59:30.194652 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.194546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21248380-e684-4a60-b641-fc605527e60c-tmp\") pod \"kube-auth-proxy-565557bd5f-xpngg\" (UID: \"21248380-e684-4a60-b641-fc605527e60c\") " pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" Jul 23 09:59:30.295775 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.295738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4s7z\" (UniqueName: \"kubernetes.io/projected/21248380-e684-4a60-b641-fc605527e60c-kube-api-access-x4s7z\") pod \"kube-auth-proxy-565557bd5f-xpngg\" (UID: \"21248380-e684-4a60-b641-fc605527e60c\") " pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" Jul 23 09:59:30.295775 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.295782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21248380-e684-4a60-b641-fc605527e60c-tmp\") pod \"kube-auth-proxy-565557bd5f-xpngg\" (UID: \"21248380-e684-4a60-b641-fc605527e60c\") " pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" Jul 23 09:59:30.296056 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.295835 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21248380-e684-4a60-b641-fc605527e60c-tls-certs\") pod \"kube-auth-proxy-565557bd5f-xpngg\" (UID: \"21248380-e684-4a60-b641-fc605527e60c\") " pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" Jul 23 09:59:30.298401 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.298372 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21248380-e684-4a60-b641-fc605527e60c-tmp\") pod \"kube-auth-proxy-565557bd5f-xpngg\" (UID: \"21248380-e684-4a60-b641-fc605527e60c\") " pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" Jul 23 09:59:30.298586 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.298569 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21248380-e684-4a60-b641-fc605527e60c-tls-certs\") pod \"kube-auth-proxy-565557bd5f-xpngg\" (UID: \"21248380-e684-4a60-b641-fc605527e60c\") " pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" Jul 23 09:59:30.305721 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.305695 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4s7z\" (UniqueName: \"kubernetes.io/projected/21248380-e684-4a60-b641-fc605527e60c-kube-api-access-x4s7z\") pod \"kube-auth-proxy-565557bd5f-xpngg\" (UID: \"21248380-e684-4a60-b641-fc605527e60c\") " pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" Jul 23 09:59:30.342730 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.342636 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" Jul 23 09:59:30.504239 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.504210 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-565557bd5f-xpngg"] Jul 23 09:59:30.506648 ip-10-0-139-55 kubenswrapper[2580]: W0723 09:59:30.506609 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21248380_e684_4a60_b641_fc605527e60c.slice/crio-e8e61a8f4e9e5895111aef34f2696d5442e55f7350aa7b893a9e1fadf2b0c5ba WatchSource:0}: Error finding container e8e61a8f4e9e5895111aef34f2696d5442e55f7350aa7b893a9e1fadf2b0c5ba: Status 404 returned error can't find the container with id e8e61a8f4e9e5895111aef34f2696d5442e55f7350aa7b893a9e1fadf2b0c5ba Jul 23 09:59:30.625899 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.625812 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" event={"ID":"21248380-e684-4a60-b641-fc605527e60c","Type":"ContainerStarted","Data":"e8e61a8f4e9e5895111aef34f2696d5442e55f7350aa7b893a9e1fadf2b0c5ba"} Jul 23 09:59:30.627498 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.627470 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" event={"ID":"2a5d3298-81cb-40f8-8dab-62d57711652c","Type":"ContainerStarted","Data":"0f201270f99e7f92a1abcb8dff950c487e95ab6f9a34a496f200a4009dd7094b"} Jul 23 09:59:30.627662 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.627599 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 09:59:30.658174 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:30.658117 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" podStartSLOduration=2.068285466 podStartE2EDuration="3.658091909s" totalImagesPullingTime="1.589806443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:59:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:59:30.65672308 +0000 UTC m=+290.903679346" watchObservedRunningTime="2026-07-23 09:59:30.658091909 +0000 UTC m=+290.905048175" Jul 23 09:59:33.645334 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:33.645293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" event={"ID":"21248380-e684-4a60-b641-fc605527e60c","Type":"ContainerStarted","Data":"877c8f428abdba306e9b75e1e15951a8b4d4a929d5b66a06606cee2d341e4faa"} Jul 23 09:59:33.662021 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:33.661951 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-565557bd5f-xpngg" podStartSLOduration=1.146104932 podStartE2EDuration="3.661928999s" totalImagesPullingTime="2.515824067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:59:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:59:33.659676513 +0000 UTC m=+293.906632780" watchObservedRunningTime="2026-07-23 09:59:33.661928999 +0000 UTC m=+293.908885265" Jul 23 09:59:40.158505 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:40.158474 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 09:59:40.162580 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:40.162557 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 09:59:40.163751 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:40.163733 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 09:59:41.634234 ip-10-0-139-55 kubenswrapper[2580]: I0723 09:59:41.634180 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d6fb6dcb8-8jmw5" Jul 23 10:00:46.595891 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.595812 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd"] Jul 23 10:00:46.598219 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.598177 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:46.600308 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.600283 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8gbj9\"" Jul 23 10:00:46.600420 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.600318 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 10:00:46.600812 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.600793 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 10:00:46.606671 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.606648 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd"] Jul 23 10:00:46.732900 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.732864 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:46.732900 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.732904 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:46.733115 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.732967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57858\" (UniqueName: \"kubernetes.io/projected/f9fcb11b-d40f-4a55-b464-b8566329b086-kube-api-access-57858\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:46.833669 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.833625 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:46.833669 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.833668 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:46.833880 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.833704 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57858\" (UniqueName: \"kubernetes.io/projected/f9fcb11b-d40f-4a55-b464-b8566329b086-kube-api-access-57858\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:46.834139 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.834103 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:46.834177 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.834128 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:46.841336 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.841310 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57858\" (UniqueName: \"kubernetes.io/projected/f9fcb11b-d40f-4a55-b464-b8566329b086-kube-api-access-57858\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:46.908368 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.908270 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:46.999558 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:46.999525 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd"] Jul 23 10:00:47.002772 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.002752 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:47.009381 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.009353 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd"] Jul 23 10:00:47.063400 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.063367 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd"] Jul 23 10:00:47.066776 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:00:47.066746 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9fcb11b_d40f_4a55_b464_b8566329b086.slice/crio-6ad8361e2aeba2da4175edbcaad0057d417dcd5d3c76886b8039eb978aec493f WatchSource:0}: Error finding container 6ad8361e2aeba2da4175edbcaad0057d417dcd5d3c76886b8039eb978aec493f: Status 404 returned error can't find the container with id 6ad8361e2aeba2da4175edbcaad0057d417dcd5d3c76886b8039eb978aec493f Jul 23 10:00:47.136296 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.136260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:47.136296 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.136303 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:47.136576 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.136324 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bckx2\" (UniqueName: \"kubernetes.io/projected/9bb321af-33a1-4857-9e9b-060fa06aaa3f-kube-api-access-bckx2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:47.237092 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.237051 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:47.237092 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.237095 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:47.237380 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.237113 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bckx2\" (UniqueName: \"kubernetes.io/projected/9bb321af-33a1-4857-9e9b-060fa06aaa3f-kube-api-access-bckx2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:47.237549 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.237526 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:47.237620 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.237584 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:47.245031 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.245004 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bckx2\" (UniqueName: \"kubernetes.io/projected/9bb321af-33a1-4857-9e9b-060fa06aaa3f-kube-api-access-bckx2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:47.315130 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.315095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:47.396628 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.396595 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl"] Jul 23 10:00:47.400078 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.400055 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:47.406623 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.406557 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl"] Jul 23 10:00:47.474955 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.474929 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd"] Jul 23 10:00:47.475857 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:00:47.475837 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb321af_33a1_4857_9e9b_060fa06aaa3f.slice/crio-0b7ff4c8638506d11f14260620b16a12c5cb761aa708c15ba1341b1d1c35b4ae WatchSource:0}: Error finding container 0b7ff4c8638506d11f14260620b16a12c5cb761aa708c15ba1341b1d1c35b4ae: Status 404 returned error can't find the container with id 0b7ff4c8638506d11f14260620b16a12c5cb761aa708c15ba1341b1d1c35b4ae Jul 23 10:00:47.541797 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.541752 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:47.541997 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.541819 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:47.541997 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.541879 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q77pc\" (UniqueName: \"kubernetes.io/projected/0f567380-cc4e-4790-acac-7bc69e32816b-kube-api-access-q77pc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:47.642858 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.642812 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:47.642858 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.642857 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q77pc\" (UniqueName: \"kubernetes.io/projected/0f567380-cc4e-4790-acac-7bc69e32816b-kube-api-access-q77pc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:47.643325 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.642899 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:47.643325 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.643268 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:47.643325 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.643308 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:47.650778 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.650745 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q77pc\" (UniqueName: \"kubernetes.io/projected/0f567380-cc4e-4790-acac-7bc69e32816b-kube-api-access-q77pc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:47.717846 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.717815 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:47.796165 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.796132 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z"] Jul 23 10:00:47.799682 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.799659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:47.807657 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.807622 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z"] Jul 23 10:00:47.860418 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.860391 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl"] Jul 23 10:00:47.862633 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:00:47.862607 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f567380_cc4e_4790_acac_7bc69e32816b.slice/crio-bc6f3599b41e48ccdfe8efba7902288510c902dc128300a2cf7969df1eec7d20 WatchSource:0}: Error finding container bc6f3599b41e48ccdfe8efba7902288510c902dc128300a2cf7969df1eec7d20: Status 404 returned error can't find the container with id bc6f3599b41e48ccdfe8efba7902288510c902dc128300a2cf7969df1eec7d20 Jul 23 10:00:47.945576 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.945544 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg7g4\" (UniqueName: \"kubernetes.io/projected/6d95e9ed-7a0f-435f-b671-ad119c756bfa-kube-api-access-rg7g4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:47.945748 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.945606 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:47.945748 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.945674 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:47.978786 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.978751 2580 generic.go:356] "Generic (PLEG): container finished" podID="0f567380-cc4e-4790-acac-7bc69e32816b" containerID="e11d23198e7a7ca1239836b9764377a1ca2d9af1804a1f35b93ac8c670bd1116" exitCode=0 Jul 23 10:00:47.978935 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.978828 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" event={"ID":"0f567380-cc4e-4790-acac-7bc69e32816b","Type":"ContainerDied","Data":"e11d23198e7a7ca1239836b9764377a1ca2d9af1804a1f35b93ac8c670bd1116"} Jul 23 10:00:47.978935 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.978881 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" event={"ID":"0f567380-cc4e-4790-acac-7bc69e32816b","Type":"ContainerStarted","Data":"bc6f3599b41e48ccdfe8efba7902288510c902dc128300a2cf7969df1eec7d20"} Jul 23 10:00:47.979424 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.979410 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 10:00:47.980309 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.980286 2580 generic.go:356] "Generic (PLEG): container finished" podID="9bb321af-33a1-4857-9e9b-060fa06aaa3f" containerID="8a8a87d8487c4e7da36ef87a7afa4cedbb1bb6e605afb48e37469c27580de1b4" exitCode=0 Jul 23 10:00:47.980398 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.980368 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" event={"ID":"9bb321af-33a1-4857-9e9b-060fa06aaa3f","Type":"ContainerDied","Data":"8a8a87d8487c4e7da36ef87a7afa4cedbb1bb6e605afb48e37469c27580de1b4"} Jul 23 10:00:47.980447 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.980406 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" event={"ID":"9bb321af-33a1-4857-9e9b-060fa06aaa3f","Type":"ContainerStarted","Data":"0b7ff4c8638506d11f14260620b16a12c5cb761aa708c15ba1341b1d1c35b4ae"} Jul 23 10:00:47.981879 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.981857 2580 generic.go:356] "Generic (PLEG): container finished" podID="f9fcb11b-d40f-4a55-b464-b8566329b086" containerID="d10b8dbc688b807a4f192e554cf9ba6e4b1d88cf56d47eb20994322757525b0e" exitCode=0 Jul 23 10:00:47.981955 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.981912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" event={"ID":"f9fcb11b-d40f-4a55-b464-b8566329b086","Type":"ContainerDied","Data":"d10b8dbc688b807a4f192e554cf9ba6e4b1d88cf56d47eb20994322757525b0e"} Jul 23 10:00:47.981955 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:47.981948 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" event={"ID":"f9fcb11b-d40f-4a55-b464-b8566329b086","Type":"ContainerStarted","Data":"6ad8361e2aeba2da4175edbcaad0057d417dcd5d3c76886b8039eb978aec493f"} Jul 23 10:00:48.046406 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.046365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg7g4\" (UniqueName: \"kubernetes.io/projected/6d95e9ed-7a0f-435f-b671-ad119c756bfa-kube-api-access-rg7g4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:48.046600 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.046436 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:48.046600 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.046461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:48.046807 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.046792 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:48.046852 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.046824 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:48.054869 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.054847 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg7g4\" (UniqueName: \"kubernetes.io/projected/6d95e9ed-7a0f-435f-b671-ad119c756bfa-kube-api-access-rg7g4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:48.111040 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.110988 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:48.247169 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.247046 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z"] Jul 23 10:00:48.249363 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:00:48.249339 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d95e9ed_7a0f_435f_b671_ad119c756bfa.slice/crio-e6ec445566c9b4b7ab566497e37106396232dc4fb6167245f9073f3f439161ba WatchSource:0}: Error finding container e6ec445566c9b4b7ab566497e37106396232dc4fb6167245f9073f3f439161ba: Status 404 returned error can't find the container with id e6ec445566c9b4b7ab566497e37106396232dc4fb6167245f9073f3f439161ba Jul 23 10:00:48.988753 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.988717 2580 generic.go:356] "Generic (PLEG): container finished" podID="0f567380-cc4e-4790-acac-7bc69e32816b" containerID="06ee3d86ba23180bae5918bf2e5d88164a0b575303e7697c8b199844c79d489e" exitCode=0 Jul 23 10:00:48.989226 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.988816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" event={"ID":"0f567380-cc4e-4790-acac-7bc69e32816b","Type":"ContainerDied","Data":"06ee3d86ba23180bae5918bf2e5d88164a0b575303e7697c8b199844c79d489e"} Jul 23 10:00:48.990639 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.990613 2580 generic.go:356] "Generic (PLEG): container finished" podID="9bb321af-33a1-4857-9e9b-060fa06aaa3f" containerID="503e7ca3736f5621e06fdf84b1e75ef18095ee3458eeb8599799218cb9437928" exitCode=0 Jul 23 10:00:48.990724 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.990701 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" event={"ID":"9bb321af-33a1-4857-9e9b-060fa06aaa3f","Type":"ContainerDied","Data":"503e7ca3736f5621e06fdf84b1e75ef18095ee3458eeb8599799218cb9437928"} Jul 23 10:00:48.992122 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.992101 2580 generic.go:356] "Generic (PLEG): container finished" podID="6d95e9ed-7a0f-435f-b671-ad119c756bfa" containerID="3dd9a98a115abb7728456190523f917a06319e5e39fe270da7207837a44df380" exitCode=0 Jul 23 10:00:48.992228 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.992182 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" event={"ID":"6d95e9ed-7a0f-435f-b671-ad119c756bfa","Type":"ContainerDied","Data":"3dd9a98a115abb7728456190523f917a06319e5e39fe270da7207837a44df380"} Jul 23 10:00:48.992329 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:48.992252 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" event={"ID":"6d95e9ed-7a0f-435f-b671-ad119c756bfa","Type":"ContainerStarted","Data":"e6ec445566c9b4b7ab566497e37106396232dc4fb6167245f9073f3f439161ba"} Jul 23 10:00:50.004523 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:50.004489 2580 generic.go:356] "Generic (PLEG): container finished" podID="0f567380-cc4e-4790-acac-7bc69e32816b" containerID="b2ce60d5245c49284f19573043dacf3c06bdae43f715dd2b01b94cde0a880680" exitCode=0 Jul 23 10:00:50.005080 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:50.004565 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" event={"ID":"0f567380-cc4e-4790-acac-7bc69e32816b","Type":"ContainerDied","Data":"b2ce60d5245c49284f19573043dacf3c06bdae43f715dd2b01b94cde0a880680"} Jul 23 10:00:50.006563 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:50.006544 2580 generic.go:356] "Generic (PLEG): container finished" podID="9bb321af-33a1-4857-9e9b-060fa06aaa3f" containerID="0a509f8c81041311010f7a502eae36feedd483e1ef8e0a41963973375323c0df" exitCode=0 Jul 23 10:00:50.006649 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:50.006606 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" event={"ID":"9bb321af-33a1-4857-9e9b-060fa06aaa3f","Type":"ContainerDied","Data":"0a509f8c81041311010f7a502eae36feedd483e1ef8e0a41963973375323c0df"} Jul 23 10:00:50.008231 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:50.008206 2580 generic.go:356] "Generic (PLEG): container finished" podID="f9fcb11b-d40f-4a55-b464-b8566329b086" containerID="ecab9164f46cabd099c39ba0a67d39d0e2cbff1e389f377442c3296426a34814" exitCode=0 Jul 23 10:00:50.008333 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:50.008275 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" event={"ID":"f9fcb11b-d40f-4a55-b464-b8566329b086","Type":"ContainerDied","Data":"ecab9164f46cabd099c39ba0a67d39d0e2cbff1e389f377442c3296426a34814"} Jul 23 10:00:50.010079 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:50.010061 2580 generic.go:356] "Generic (PLEG): container finished" podID="6d95e9ed-7a0f-435f-b671-ad119c756bfa" containerID="a2bdb02ef041c8d6b8752edcd22ecc26e8724af0b432cb2e023c8741631ee8a1" exitCode=0 Jul 23 10:00:50.010160 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:50.010127 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" event={"ID":"6d95e9ed-7a0f-435f-b671-ad119c756bfa","Type":"ContainerDied","Data":"a2bdb02ef041c8d6b8752edcd22ecc26e8724af0b432cb2e023c8741631ee8a1"} Jul 23 10:00:51.016881 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.016848 2580 generic.go:356] "Generic (PLEG): container finished" podID="f9fcb11b-d40f-4a55-b464-b8566329b086" containerID="d98dbeb484b67f3f1107cfe526522b91eb9f23d4abd8af10bddbd50ad0343bba" exitCode=0 Jul 23 10:00:51.017341 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.016937 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" event={"ID":"f9fcb11b-d40f-4a55-b464-b8566329b086","Type":"ContainerDied","Data":"d98dbeb484b67f3f1107cfe526522b91eb9f23d4abd8af10bddbd50ad0343bba"} Jul 23 10:00:51.018944 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.018922 2580 generic.go:356] "Generic (PLEG): container finished" podID="6d95e9ed-7a0f-435f-b671-ad119c756bfa" containerID="15b68b2445c675f67288d17d738a0fee54eaab2dc5b6046c4c6d1c4fe3d5b9a0" exitCode=0 Jul 23 10:00:51.019068 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.018954 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" event={"ID":"6d95e9ed-7a0f-435f-b671-ad119c756bfa","Type":"ContainerDied","Data":"15b68b2445c675f67288d17d738a0fee54eaab2dc5b6046c4c6d1c4fe3d5b9a0"} Jul 23 10:00:51.186213 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.186150 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:51.189002 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.188978 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:51.272044 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.271945 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-bundle\") pod \"0f567380-cc4e-4790-acac-7bc69e32816b\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " Jul 23 10:00:51.272044 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.271993 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bckx2\" (UniqueName: \"kubernetes.io/projected/9bb321af-33a1-4857-9e9b-060fa06aaa3f-kube-api-access-bckx2\") pod \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " Jul 23 10:00:51.272044 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.272009 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-util\") pod \"0f567380-cc4e-4790-acac-7bc69e32816b\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " Jul 23 10:00:51.272044 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.272048 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q77pc\" (UniqueName: \"kubernetes.io/projected/0f567380-cc4e-4790-acac-7bc69e32816b-kube-api-access-q77pc\") pod \"0f567380-cc4e-4790-acac-7bc69e32816b\" (UID: \"0f567380-cc4e-4790-acac-7bc69e32816b\") " Jul 23 10:00:51.272415 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.272106 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-bundle\") pod \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " Jul 23 10:00:51.272415 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.272120 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-util\") pod \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\" (UID: \"9bb321af-33a1-4857-9e9b-060fa06aaa3f\") " Jul 23 10:00:51.272633 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.272600 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-bundle" (OuterVolumeSpecName: "bundle") pod "0f567380-cc4e-4790-acac-7bc69e32816b" (UID: "0f567380-cc4e-4790-acac-7bc69e32816b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:00:51.272775 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.272749 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-bundle" (OuterVolumeSpecName: "bundle") pod "9bb321af-33a1-4857-9e9b-060fa06aaa3f" (UID: "9bb321af-33a1-4857-9e9b-060fa06aaa3f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:00:51.275028 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.274989 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bb321af-33a1-4857-9e9b-060fa06aaa3f-kube-api-access-bckx2" (OuterVolumeSpecName: "kube-api-access-bckx2") pod "9bb321af-33a1-4857-9e9b-060fa06aaa3f" (UID: "9bb321af-33a1-4857-9e9b-060fa06aaa3f"). InnerVolumeSpecName "kube-api-access-bckx2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:00:51.275142 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.275021 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f567380-cc4e-4790-acac-7bc69e32816b-kube-api-access-q77pc" (OuterVolumeSpecName: "kube-api-access-q77pc") pod "0f567380-cc4e-4790-acac-7bc69e32816b" (UID: "0f567380-cc4e-4790-acac-7bc69e32816b"). InnerVolumeSpecName "kube-api-access-q77pc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:00:51.278761 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.278725 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-util" (OuterVolumeSpecName: "util") pod "9bb321af-33a1-4857-9e9b-060fa06aaa3f" (UID: "9bb321af-33a1-4857-9e9b-060fa06aaa3f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:00:51.278922 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.278894 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-util" (OuterVolumeSpecName: "util") pod "0f567380-cc4e-4790-acac-7bc69e32816b" (UID: "0f567380-cc4e-4790-acac-7bc69e32816b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:00:51.372877 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.372835 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-bundle\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:51.372877 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.372870 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bb321af-33a1-4857-9e9b-060fa06aaa3f-util\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:51.372877 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.372883 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-bundle\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:51.373116 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.372895 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bckx2\" (UniqueName: \"kubernetes.io/projected/9bb321af-33a1-4857-9e9b-060fa06aaa3f-kube-api-access-bckx2\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:51.373116 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.372908 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f567380-cc4e-4790-acac-7bc69e32816b-util\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:51.373116 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:51.372919 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q77pc\" (UniqueName: \"kubernetes.io/projected/0f567380-cc4e-4790-acac-7bc69e32816b-kube-api-access-q77pc\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:52.026124 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.026082 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" event={"ID":"0f567380-cc4e-4790-acac-7bc69e32816b","Type":"ContainerDied","Data":"bc6f3599b41e48ccdfe8efba7902288510c902dc128300a2cf7969df1eec7d20"} Jul 23 10:00:52.026550 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.026142 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc6f3599b41e48ccdfe8efba7902288510c902dc128300a2cf7969df1eec7d20" Jul 23 10:00:52.026550 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.026097 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl" Jul 23 10:00:52.028101 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.028080 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" Jul 23 10:00:52.028101 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.028082 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd" event={"ID":"9bb321af-33a1-4857-9e9b-060fa06aaa3f","Type":"ContainerDied","Data":"0b7ff4c8638506d11f14260620b16a12c5cb761aa708c15ba1341b1d1c35b4ae"} Jul 23 10:00:52.028292 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.028139 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b7ff4c8638506d11f14260620b16a12c5cb761aa708c15ba1341b1d1c35b4ae" Jul 23 10:00:52.189264 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.189239 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:52.191049 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.191032 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:52.279117 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.279027 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57858\" (UniqueName: \"kubernetes.io/projected/f9fcb11b-d40f-4a55-b464-b8566329b086-kube-api-access-57858\") pod \"f9fcb11b-d40f-4a55-b464-b8566329b086\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " Jul 23 10:00:52.279117 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.279065 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-util\") pod \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " Jul 23 10:00:52.279380 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.279142 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-util\") pod \"f9fcb11b-d40f-4a55-b464-b8566329b086\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " Jul 23 10:00:52.279380 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.279175 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg7g4\" (UniqueName: \"kubernetes.io/projected/6d95e9ed-7a0f-435f-b671-ad119c756bfa-kube-api-access-rg7g4\") pod \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " Jul 23 10:00:52.279380 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.279229 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-bundle\") pod \"f9fcb11b-d40f-4a55-b464-b8566329b086\" (UID: \"f9fcb11b-d40f-4a55-b464-b8566329b086\") " Jul 23 10:00:52.279380 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.279257 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-bundle\") pod \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\" (UID: \"6d95e9ed-7a0f-435f-b671-ad119c756bfa\") " Jul 23 10:00:52.280075 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.280040 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-bundle" (OuterVolumeSpecName: "bundle") pod "f9fcb11b-d40f-4a55-b464-b8566329b086" (UID: "f9fcb11b-d40f-4a55-b464-b8566329b086"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:00:52.280075 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.280042 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-bundle" (OuterVolumeSpecName: "bundle") pod "6d95e9ed-7a0f-435f-b671-ad119c756bfa" (UID: "6d95e9ed-7a0f-435f-b671-ad119c756bfa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:00:52.281501 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.281477 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9fcb11b-d40f-4a55-b464-b8566329b086-kube-api-access-57858" (OuterVolumeSpecName: "kube-api-access-57858") pod "f9fcb11b-d40f-4a55-b464-b8566329b086" (UID: "f9fcb11b-d40f-4a55-b464-b8566329b086"). InnerVolumeSpecName "kube-api-access-57858". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:00:52.281689 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.281665 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d95e9ed-7a0f-435f-b671-ad119c756bfa-kube-api-access-rg7g4" (OuterVolumeSpecName: "kube-api-access-rg7g4") pod "6d95e9ed-7a0f-435f-b671-ad119c756bfa" (UID: "6d95e9ed-7a0f-435f-b671-ad119c756bfa"). InnerVolumeSpecName "kube-api-access-rg7g4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:00:52.284888 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.284850 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-util" (OuterVolumeSpecName: "util") pod "6d95e9ed-7a0f-435f-b671-ad119c756bfa" (UID: "6d95e9ed-7a0f-435f-b671-ad119c756bfa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:00:52.371120 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.371074 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-util" (OuterVolumeSpecName: "util") pod "f9fcb11b-d40f-4a55-b464-b8566329b086" (UID: "f9fcb11b-d40f-4a55-b464-b8566329b086"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:00:52.379950 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.379921 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57858\" (UniqueName: \"kubernetes.io/projected/f9fcb11b-d40f-4a55-b464-b8566329b086-kube-api-access-57858\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:52.379950 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.379951 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-util\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:52.380084 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.379963 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-util\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:52.380084 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.379971 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg7g4\" (UniqueName: \"kubernetes.io/projected/6d95e9ed-7a0f-435f-b671-ad119c756bfa-kube-api-access-rg7g4\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:52.380084 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.379980 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9fcb11b-d40f-4a55-b464-b8566329b086-bundle\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:52.380084 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:52.379988 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d95e9ed-7a0f-435f-b671-ad119c756bfa-bundle\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:00:53.034564 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:53.034531 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" event={"ID":"f9fcb11b-d40f-4a55-b464-b8566329b086","Type":"ContainerDied","Data":"6ad8361e2aeba2da4175edbcaad0057d417dcd5d3c76886b8039eb978aec493f"} Jul 23 10:00:53.035040 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:53.034583 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ad8361e2aeba2da4175edbcaad0057d417dcd5d3c76886b8039eb978aec493f" Jul 23 10:00:53.035040 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:53.034544 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd" Jul 23 10:00:53.036379 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:53.036348 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" Jul 23 10:00:53.036379 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:53.036354 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z" event={"ID":"6d95e9ed-7a0f-435f-b671-ad119c756bfa","Type":"ContainerDied","Data":"e6ec445566c9b4b7ab566497e37106396232dc4fb6167245f9073f3f439161ba"} Jul 23 10:00:53.036545 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:00:53.036401 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6ec445566c9b4b7ab566497e37106396232dc4fb6167245f9073f3f439161ba" Jul 23 10:01:18.351300 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351263 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c57fb7b78-gchrv"] Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351591 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f567380-cc4e-4790-acac-7bc69e32816b" containerName="util" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351604 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f567380-cc4e-4790-acac-7bc69e32816b" containerName="util" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351613 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb321af-33a1-4857-9e9b-060fa06aaa3f" containerName="pull" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351619 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb321af-33a1-4857-9e9b-060fa06aaa3f" containerName="pull" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351625 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9fcb11b-d40f-4a55-b464-b8566329b086" containerName="util" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351631 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9fcb11b-d40f-4a55-b464-b8566329b086" containerName="util" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351643 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9fcb11b-d40f-4a55-b464-b8566329b086" containerName="pull" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351648 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9fcb11b-d40f-4a55-b464-b8566329b086" containerName="pull" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351655 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9fcb11b-d40f-4a55-b464-b8566329b086" containerName="extract" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351660 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9fcb11b-d40f-4a55-b464-b8566329b086" containerName="extract" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351668 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d95e9ed-7a0f-435f-b671-ad119c756bfa" containerName="util" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351672 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d95e9ed-7a0f-435f-b671-ad119c756bfa" containerName="util" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351678 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f567380-cc4e-4790-acac-7bc69e32816b" containerName="extract" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351683 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f567380-cc4e-4790-acac-7bc69e32816b" containerName="extract" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351688 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d95e9ed-7a0f-435f-b671-ad119c756bfa" containerName="extract" Jul 23 10:01:18.351685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351693 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d95e9ed-7a0f-435f-b671-ad119c756bfa" containerName="extract" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351700 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb321af-33a1-4857-9e9b-060fa06aaa3f" containerName="extract" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351704 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb321af-33a1-4857-9e9b-060fa06aaa3f" containerName="extract" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351710 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d95e9ed-7a0f-435f-b671-ad119c756bfa" containerName="pull" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351714 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d95e9ed-7a0f-435f-b671-ad119c756bfa" containerName="pull" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351721 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f567380-cc4e-4790-acac-7bc69e32816b" containerName="pull" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351726 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f567380-cc4e-4790-acac-7bc69e32816b" containerName="pull" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351733 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb321af-33a1-4857-9e9b-060fa06aaa3f" containerName="util" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351738 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb321af-33a1-4857-9e9b-060fa06aaa3f" containerName="util" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351787 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f567380-cc4e-4790-acac-7bc69e32816b" containerName="extract" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351795 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bb321af-33a1-4857-9e9b-060fa06aaa3f" containerName="extract" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351801 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9fcb11b-d40f-4a55-b464-b8566329b086" containerName="extract" Jul 23 10:01:18.352137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.351806 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d95e9ed-7a0f-435f-b671-ad119c756bfa" containerName="extract" Jul 23 10:01:18.356613 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.356588 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.358655 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.358632 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 10:01:18.359265 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.359234 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 10:01:18.359393 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.359312 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 10:01:18.359393 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.359335 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 10:01:18.359393 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.359343 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2wlzg\"" Jul 23 10:01:18.359393 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.359383 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 10:01:18.365241 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.365178 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 10:01:18.366036 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.366008 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c57fb7b78-gchrv"] Jul 23 10:01:18.486036 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.485992 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-console-oauth-config\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.486265 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.486051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-console-serving-cert\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.486265 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.486119 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-trusted-ca-bundle\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.486265 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.486218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-console-config\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.486265 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.486258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-oauth-serving-cert\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.486460 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.486291 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5wms\" (UniqueName: \"kubernetes.io/projected/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-kube-api-access-c5wms\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.486460 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.486344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-service-ca\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.587477 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.587430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-console-config\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.587477 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.587477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-oauth-serving-cert\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.587753 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.587501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5wms\" (UniqueName: \"kubernetes.io/projected/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-kube-api-access-c5wms\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.587753 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.587516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-service-ca\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.587753 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.587537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-console-oauth-config\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.587753 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.587566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-console-serving-cert\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.587753 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.587584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-trusted-ca-bundle\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.588353 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.588315 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-service-ca\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.588483 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.588359 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-oauth-serving-cert\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.588483 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.588368 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-console-config\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.588483 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.588468 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-trusted-ca-bundle\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.590314 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.590296 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-console-oauth-config\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.590411 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.590393 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-console-serving-cert\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.598180 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.598153 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5wms\" (UniqueName: \"kubernetes.io/projected/1951a0ec-3156-49cf-8fb1-9ec47d86fbd5-kube-api-access-c5wms\") pod \"console-7c57fb7b78-gchrv\" (UID: \"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5\") " pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.667453 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.667352 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:18.813469 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:18.813440 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c57fb7b78-gchrv"] Jul 23 10:01:18.815432 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:01:18.815399 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1951a0ec_3156_49cf_8fb1_9ec47d86fbd5.slice/crio-8287410d8c3b76848e2217726fe285ff13f62ee94f2217fba180b7244bf06bd8 WatchSource:0}: Error finding container 8287410d8c3b76848e2217726fe285ff13f62ee94f2217fba180b7244bf06bd8: Status 404 returned error can't find the container with id 8287410d8c3b76848e2217726fe285ff13f62ee94f2217fba180b7244bf06bd8 Jul 23 10:01:19.156809 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:19.156774 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c57fb7b78-gchrv" event={"ID":"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5","Type":"ContainerStarted","Data":"3969bea6b2cb459b9318fa2a2828279353b7ce18e79d9c2d103766b2c1c5ee75"} Jul 23 10:01:19.157011 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:19.156827 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c57fb7b78-gchrv" event={"ID":"1951a0ec-3156-49cf-8fb1-9ec47d86fbd5","Type":"ContainerStarted","Data":"8287410d8c3b76848e2217726fe285ff13f62ee94f2217fba180b7244bf06bd8"} Jul 23 10:01:19.177591 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:19.177530 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c57fb7b78-gchrv" podStartSLOduration=1.177513051 podStartE2EDuration="1.177513051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:01:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:01:19.175467813 +0000 UTC m=+399.422424080" watchObservedRunningTime="2026-07-23 10:01:19.177513051 +0000 UTC m=+399.424469386" Jul 23 10:01:20.560815 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.560777 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs"] Jul 23 10:01:20.564623 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.564590 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:20.566686 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.566656 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 23 10:01:20.566686 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.566674 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 10:01:20.566902 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.566729 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8gbj9\"" Jul 23 10:01:20.566902 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.566656 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 23 10:01:20.567344 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.567328 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 10:01:20.572154 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.572125 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs"] Jul 23 10:01:20.604059 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.604024 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/10496e28-71fa-4e14-b42e-d87c0b903bf4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-g4vcs\" (UID: \"10496e28-71fa-4e14-b42e-d87c0b903bf4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:20.604059 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.604073 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs2t4\" (UniqueName: \"kubernetes.io/projected/10496e28-71fa-4e14-b42e-d87c0b903bf4-kube-api-access-xs2t4\") pod \"kuadrant-console-plugin-66b5774985-g4vcs\" (UID: \"10496e28-71fa-4e14-b42e-d87c0b903bf4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:20.604323 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.604166 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/10496e28-71fa-4e14-b42e-d87c0b903bf4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-g4vcs\" (UID: \"10496e28-71fa-4e14-b42e-d87c0b903bf4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:20.705412 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.705374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs2t4\" (UniqueName: \"kubernetes.io/projected/10496e28-71fa-4e14-b42e-d87c0b903bf4-kube-api-access-xs2t4\") pod \"kuadrant-console-plugin-66b5774985-g4vcs\" (UID: \"10496e28-71fa-4e14-b42e-d87c0b903bf4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:20.705588 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.705476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/10496e28-71fa-4e14-b42e-d87c0b903bf4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-g4vcs\" (UID: \"10496e28-71fa-4e14-b42e-d87c0b903bf4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:20.705588 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.705506 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/10496e28-71fa-4e14-b42e-d87c0b903bf4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-g4vcs\" (UID: \"10496e28-71fa-4e14-b42e-d87c0b903bf4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:20.705663 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:01:20.705616 2580 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 23 10:01:20.705697 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:01:20.705685 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10496e28-71fa-4e14-b42e-d87c0b903bf4-plugin-serving-cert podName:10496e28-71fa-4e14-b42e-d87c0b903bf4 nodeName:}" failed. No retries permitted until 2026-07-23 10:01:21.205666747 +0000 UTC m=+401.452622991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/10496e28-71fa-4e14-b42e-d87c0b903bf4-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-g4vcs" (UID: "10496e28-71fa-4e14-b42e-d87c0b903bf4") : secret "plugin-serving-cert" not found Jul 23 10:01:20.706137 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.706117 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/10496e28-71fa-4e14-b42e-d87c0b903bf4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-g4vcs\" (UID: \"10496e28-71fa-4e14-b42e-d87c0b903bf4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:20.717846 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:20.717814 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs2t4\" (UniqueName: \"kubernetes.io/projected/10496e28-71fa-4e14-b42e-d87c0b903bf4-kube-api-access-xs2t4\") pod \"kuadrant-console-plugin-66b5774985-g4vcs\" (UID: \"10496e28-71fa-4e14-b42e-d87c0b903bf4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:21.210071 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:21.210026 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/10496e28-71fa-4e14-b42e-d87c0b903bf4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-g4vcs\" (UID: \"10496e28-71fa-4e14-b42e-d87c0b903bf4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:21.212716 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:21.212681 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/10496e28-71fa-4e14-b42e-d87c0b903bf4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-g4vcs\" (UID: \"10496e28-71fa-4e14-b42e-d87c0b903bf4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:21.476053 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:21.475964 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" Jul 23 10:01:21.613830 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:21.613807 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs"] Jul 23 10:01:21.615858 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:01:21.615833 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10496e28_71fa_4e14_b42e_d87c0b903bf4.slice/crio-439ce7f39eb020da1ded78434bfc035352233943a2a1bc42edc2351b66ae3fcf WatchSource:0}: Error finding container 439ce7f39eb020da1ded78434bfc035352233943a2a1bc42edc2351b66ae3fcf: Status 404 returned error can't find the container with id 439ce7f39eb020da1ded78434bfc035352233943a2a1bc42edc2351b66ae3fcf Jul 23 10:01:22.174712 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:22.174669 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" event={"ID":"10496e28-71fa-4e14-b42e-d87c0b903bf4","Type":"ContainerStarted","Data":"439ce7f39eb020da1ded78434bfc035352233943a2a1bc42edc2351b66ae3fcf"} Jul 23 10:01:28.667491 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:28.667451 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:28.667993 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:28.667544 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:28.673018 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:28.672987 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:29.231530 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:29.231492 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c57fb7b78-gchrv" Jul 23 10:01:30.685376 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:30.685338 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x"] Jul 23 10:01:30.702047 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:30.702010 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:30.704540 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:30.704319 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7jndm\"" Jul 23 10:01:30.705494 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:30.705464 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x"] Jul 23 10:01:30.800537 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:30.800491 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg2fl\" (UniqueName: \"kubernetes.io/projected/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-kube-api-access-dg2fl\") pod \"kuadrant-operator-controller-manager-55467d9f7-mmk7x\" (UID: \"c8ec64f1-13ba-4da6-a8d7-4f2ace51737b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:30.800736 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:30.800647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mmk7x\" (UID: \"c8ec64f1-13ba-4da6-a8d7-4f2ace51737b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:30.901315 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:30.901270 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg2fl\" (UniqueName: \"kubernetes.io/projected/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-kube-api-access-dg2fl\") pod \"kuadrant-operator-controller-manager-55467d9f7-mmk7x\" (UID: \"c8ec64f1-13ba-4da6-a8d7-4f2ace51737b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:30.901502 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:30.901389 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mmk7x\" (UID: \"c8ec64f1-13ba-4da6-a8d7-4f2ace51737b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:30.901801 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:30.901778 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mmk7x\" (UID: \"c8ec64f1-13ba-4da6-a8d7-4f2ace51737b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:30.909339 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:30.909307 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg2fl\" (UniqueName: \"kubernetes.io/projected/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-kube-api-access-dg2fl\") pod \"kuadrant-operator-controller-manager-55467d9f7-mmk7x\" (UID: \"c8ec64f1-13ba-4da6-a8d7-4f2ace51737b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:31.017996 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.017951 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:31.119276 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.119241 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x"] Jul 23 10:01:31.123760 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.123723 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x"] Jul 23 10:01:31.142682 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.142610 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n"] Jul 23 10:01:31.148867 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.148840 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:01:31.164932 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.164890 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n"] Jul 23 10:01:31.305642 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.305373 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-phl7n\" (UID: \"4ebca67a-352f-411b-9ec6-a3f9b6b073f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:01:31.305642 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.305427 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct2qj\" (UniqueName: \"kubernetes.io/projected/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-kube-api-access-ct2qj\") pod \"kuadrant-operator-controller-manager-55467d9f7-phl7n\" (UID: \"4ebca67a-352f-411b-9ec6-a3f9b6b073f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:01:31.406806 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.406767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-phl7n\" (UID: \"4ebca67a-352f-411b-9ec6-a3f9b6b073f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:01:31.407012 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.406841 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct2qj\" (UniqueName: \"kubernetes.io/projected/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-kube-api-access-ct2qj\") pod \"kuadrant-operator-controller-manager-55467d9f7-phl7n\" (UID: \"4ebca67a-352f-411b-9ec6-a3f9b6b073f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:01:31.407164 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.407140 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-phl7n\" (UID: \"4ebca67a-352f-411b-9ec6-a3f9b6b073f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:01:31.416504 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.416471 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct2qj\" (UniqueName: \"kubernetes.io/projected/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-kube-api-access-ct2qj\") pod \"kuadrant-operator-controller-manager-55467d9f7-phl7n\" (UID: \"4ebca67a-352f-411b-9ec6-a3f9b6b073f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:01:31.464125 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:31.464088 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:01:43.098652 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:43.098617 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n"] Jul 23 10:01:43.100495 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:01:43.100464 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ebca67a_352f_411b_9ec6_a3f9b6b073f7.slice/crio-13a26f6224b92bb8acc200e0ce8f477b30bfd6685f620f50a72d57fca60640ff WatchSource:0}: Error finding container 13a26f6224b92bb8acc200e0ce8f477b30bfd6685f620f50a72d57fca60640ff: Status 404 returned error can't find the container with id 13a26f6224b92bb8acc200e0ce8f477b30bfd6685f620f50a72d57fca60640ff Jul 23 10:01:43.303089 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:43.302989 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" event={"ID":"10496e28-71fa-4e14-b42e-d87c0b903bf4","Type":"ContainerStarted","Data":"ec7ce03319799c1766a09a6703d16a046d262dc8e2a8e3239f1d865bc302c283"} Jul 23 10:01:43.304158 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:43.304134 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" event={"ID":"4ebca67a-352f-411b-9ec6-a3f9b6b073f7","Type":"ContainerStarted","Data":"13a26f6224b92bb8acc200e0ce8f477b30bfd6685f620f50a72d57fca60640ff"} Jul 23 10:01:43.322042 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:43.321988 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g4vcs" podStartSLOduration=1.919563127 podStartE2EDuration="23.321967058s" totalImagesPullingTime="21.402403931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:01:43.318740853 +0000 UTC m=+423.565697119" watchObservedRunningTime="2026-07-23 10:01:43.321967058 +0000 UTC m=+423.568923326" Jul 23 10:01:45.502113 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:01:45.502071 2580 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-mmk7x_kuadrant-system_c8ec64f1-13ba-4da6-a8d7-4f2ace51737b_0(f182a5c7bc569cb37aef4a5e309e5b7dfa8b5e58f8a9b212f5c5202bad016733): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-mmk7x to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 23 10:01:45.502498 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:01:45.502143 2580 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-mmk7x_kuadrant-system_c8ec64f1-13ba-4da6-a8d7-4f2ace51737b_0(f182a5c7bc569cb37aef4a5e309e5b7dfa8b5e58f8a9b212f5c5202bad016733): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-mmk7x to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:46.327693 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:46.327661 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:46.329428 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:01:46.329384 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mmk7x\" is forbidden: User \"system:node:ip-10-0-139-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-55.ec2.internal' and this object" podUID="c8ec64f1-13ba-4da6-a8d7-4f2ace51737b" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:46.334925 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:46.334897 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:46.336848 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:01:46.336812 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mmk7x\" is forbidden: User \"system:node:ip-10-0-139-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-55.ec2.internal' and this object" podUID="c8ec64f1-13ba-4da6-a8d7-4f2ace51737b" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:46.445768 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:46.445708 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg2fl\" (UniqueName: \"kubernetes.io/projected/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-kube-api-access-dg2fl\") pod \"c8ec64f1-13ba-4da6-a8d7-4f2ace51737b\" (UID: \"c8ec64f1-13ba-4da6-a8d7-4f2ace51737b\") " Jul 23 10:01:46.445768 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:46.445773 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-extensions-socket-volume\") pod \"c8ec64f1-13ba-4da6-a8d7-4f2ace51737b\" (UID: \"c8ec64f1-13ba-4da6-a8d7-4f2ace51737b\") " Jul 23 10:01:46.446093 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:46.446061 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c8ec64f1-13ba-4da6-a8d7-4f2ace51737b" (UID: "c8ec64f1-13ba-4da6-a8d7-4f2ace51737b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:01:46.448107 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:46.448080 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-kube-api-access-dg2fl" (OuterVolumeSpecName: "kube-api-access-dg2fl") pod "c8ec64f1-13ba-4da6-a8d7-4f2ace51737b" (UID: "c8ec64f1-13ba-4da6-a8d7-4f2ace51737b"). InnerVolumeSpecName "kube-api-access-dg2fl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:01:46.547434 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:46.547391 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg2fl\" (UniqueName: \"kubernetes.io/projected/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-kube-api-access-dg2fl\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:01:46.547434 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:46.547425 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b-extensions-socket-volume\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:01:47.334184 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:47.334153 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:47.336062 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:01:47.336028 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mmk7x\" is forbidden: User \"system:node:ip-10-0-139-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-55.ec2.internal' and this object" podUID="c8ec64f1-13ba-4da6-a8d7-4f2ace51737b" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:47.342848 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:01:47.342810 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mmk7x\" is forbidden: User \"system:node:ip-10-0-139-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-55.ec2.internal' and this object" podUID="c8ec64f1-13ba-4da6-a8d7-4f2ace51737b" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mmk7x" Jul 23 10:01:48.279973 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:48.279935 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8ec64f1-13ba-4da6-a8d7-4f2ace51737b" path="/var/lib/kubelet/pods/c8ec64f1-13ba-4da6-a8d7-4f2ace51737b/volumes" Jul 23 10:01:48.340880 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:48.340845 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" event={"ID":"4ebca67a-352f-411b-9ec6-a3f9b6b073f7","Type":"ContainerStarted","Data":"c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092"} Jul 23 10:01:48.341070 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:48.340937 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:01:48.362127 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:48.362072 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" podStartSLOduration=12.589305069 podStartE2EDuration="17.362055423s" totalImagesPullingTime="4.772750354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:01:48.359468898 +0000 UTC m=+428.606425176" watchObservedRunningTime="2026-07-23 10:01:48.362055423 +0000 UTC m=+428.609011688" Jul 23 10:01:59.348262 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:01:59.348232 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:02:11.630938 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:11.630856 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n"] Jul 23 10:02:11.631556 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:11.631088 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" podUID="4ebca67a-352f-411b-9ec6-a3f9b6b073f7" containerName="manager" containerID="cri-o://c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092" gracePeriod=10 Jul 23 10:02:12.452900 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.452863 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:02:12.463250 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.463212 2580 generic.go:356] "Generic (PLEG): container finished" podID="4ebca67a-352f-411b-9ec6-a3f9b6b073f7" containerID="c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092" exitCode=0 Jul 23 10:02:12.463402 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.463253 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" Jul 23 10:02:12.463402 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.463288 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" event={"ID":"4ebca67a-352f-411b-9ec6-a3f9b6b073f7","Type":"ContainerDied","Data":"c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092"} Jul 23 10:02:12.463402 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.463350 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n" event={"ID":"4ebca67a-352f-411b-9ec6-a3f9b6b073f7","Type":"ContainerDied","Data":"13a26f6224b92bb8acc200e0ce8f477b30bfd6685f620f50a72d57fca60640ff"} Jul 23 10:02:12.463402 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.463380 2580 scope.go:117] "RemoveContainer" containerID="c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092" Jul 23 10:02:12.474752 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.474734 2580 scope.go:117] "RemoveContainer" containerID="c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092" Jul 23 10:02:12.475117 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:02:12.475095 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092\": container with ID starting with c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092 not found: ID does not exist" containerID="c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092" Jul 23 10:02:12.475222 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.475128 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092"} err="failed to get container status \"c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092\": rpc error: code = NotFound desc = could not find container \"c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092\": container with ID starting with c1e6e7521c13d4fcca2d4f68eb9aad6f051efadec47d08469d10cead3b66a092 not found: ID does not exist" Jul 23 10:02:12.558773 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.558675 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-extensions-socket-volume\") pod \"4ebca67a-352f-411b-9ec6-a3f9b6b073f7\" (UID: \"4ebca67a-352f-411b-9ec6-a3f9b6b073f7\") " Jul 23 10:02:12.558773 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.558721 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ct2qj\" (UniqueName: \"kubernetes.io/projected/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-kube-api-access-ct2qj\") pod \"4ebca67a-352f-411b-9ec6-a3f9b6b073f7\" (UID: \"4ebca67a-352f-411b-9ec6-a3f9b6b073f7\") " Jul 23 10:02:12.559251 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.559224 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4ebca67a-352f-411b-9ec6-a3f9b6b073f7" (UID: "4ebca67a-352f-411b-9ec6-a3f9b6b073f7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:02:12.560951 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.560924 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-kube-api-access-ct2qj" (OuterVolumeSpecName: "kube-api-access-ct2qj") pod "4ebca67a-352f-411b-9ec6-a3f9b6b073f7" (UID: "4ebca67a-352f-411b-9ec6-a3f9b6b073f7"). InnerVolumeSpecName "kube-api-access-ct2qj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:02:12.659462 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.659423 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-extensions-socket-volume\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:02:12.659462 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.659458 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ct2qj\" (UniqueName: \"kubernetes.io/projected/4ebca67a-352f-411b-9ec6-a3f9b6b073f7-kube-api-access-ct2qj\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:02:12.784123 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.784090 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n"] Jul 23 10:02:12.791877 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:12.791847 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-phl7n"] Jul 23 10:02:14.279587 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:14.279553 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ebca67a-352f-411b-9ec6-a3f9b6b073f7" path="/var/lib/kubelet/pods/4ebca67a-352f-411b-9ec6-a3f9b6b073f7/volumes" Jul 23 10:02:17.930143 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:17.930103 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:02:17.930690 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:17.930428 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ebca67a-352f-411b-9ec6-a3f9b6b073f7" containerName="manager" Jul 23 10:02:17.930690 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:17.930480 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ebca67a-352f-411b-9ec6-a3f9b6b073f7" containerName="manager" Jul 23 10:02:17.930690 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:17.930548 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ebca67a-352f-411b-9ec6-a3f9b6b073f7" containerName="manager" Jul 23 10:02:17.942975 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:17.942938 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:02:17.943155 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:17.943033 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" Jul 23 10:02:17.944966 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:17.944937 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 10:02:17.974276 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:17.974244 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:02:18.007317 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.007278 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e30101a3-6590-47c8-977f-ba5d8503f742-config-file\") pod \"limitador-limitador-54d6d988d5-b5268\" (UID: \"e30101a3-6590-47c8-977f-ba5d8503f742\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" Jul 23 10:02:18.007317 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.007317 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgps6\" (UniqueName: \"kubernetes.io/projected/e30101a3-6590-47c8-977f-ba5d8503f742-kube-api-access-dgps6\") pod \"limitador-limitador-54d6d988d5-b5268\" (UID: \"e30101a3-6590-47c8-977f-ba5d8503f742\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" Jul 23 10:02:18.107924 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.107887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e30101a3-6590-47c8-977f-ba5d8503f742-config-file\") pod \"limitador-limitador-54d6d988d5-b5268\" (UID: \"e30101a3-6590-47c8-977f-ba5d8503f742\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" Jul 23 10:02:18.108130 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.107939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgps6\" (UniqueName: \"kubernetes.io/projected/e30101a3-6590-47c8-977f-ba5d8503f742-kube-api-access-dgps6\") pod \"limitador-limitador-54d6d988d5-b5268\" (UID: \"e30101a3-6590-47c8-977f-ba5d8503f742\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" Jul 23 10:02:18.108643 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.108616 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e30101a3-6590-47c8-977f-ba5d8503f742-config-file\") pod \"limitador-limitador-54d6d988d5-b5268\" (UID: \"e30101a3-6590-47c8-977f-ba5d8503f742\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" Jul 23 10:02:18.115239 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.115214 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgps6\" (UniqueName: \"kubernetes.io/projected/e30101a3-6590-47c8-977f-ba5d8503f742-kube-api-access-dgps6\") pod \"limitador-limitador-54d6d988d5-b5268\" (UID: \"e30101a3-6590-47c8-977f-ba5d8503f742\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" Jul 23 10:02:18.254077 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.254035 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" Jul 23 10:02:18.328944 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.328909 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-vvvsl"] Jul 23 10:02:18.335633 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.335608 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vvvsl" Jul 23 10:02:18.338352 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.338232 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-f6qjl\"" Jul 23 10:02:18.340085 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.340056 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vvvsl"] Jul 23 10:02:18.401401 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.401376 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:02:18.403787 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:02:18.403752 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode30101a3_6590_47c8_977f_ba5d8503f742.slice/crio-1902557ed94431cde90bc1104f43b83694a710aa0ec2224357b4f1bb62e62a9b WatchSource:0}: Error finding container 1902557ed94431cde90bc1104f43b83694a710aa0ec2224357b4f1bb62e62a9b: Status 404 returned error can't find the container with id 1902557ed94431cde90bc1104f43b83694a710aa0ec2224357b4f1bb62e62a9b Jul 23 10:02:18.411087 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.411061 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rchvp\" (UniqueName: \"kubernetes.io/projected/c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75-kube-api-access-rchvp\") pod \"authorino-58cd955675-vvvsl\" (UID: \"c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75\") " pod="kuadrant-system/authorino-58cd955675-vvvsl" Jul 23 10:02:18.494346 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.494316 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" event={"ID":"e30101a3-6590-47c8-977f-ba5d8503f742","Type":"ContainerStarted","Data":"1902557ed94431cde90bc1104f43b83694a710aa0ec2224357b4f1bb62e62a9b"} Jul 23 10:02:18.512037 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.511944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rchvp\" (UniqueName: \"kubernetes.io/projected/c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75-kube-api-access-rchvp\") pod \"authorino-58cd955675-vvvsl\" (UID: \"c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75\") " pod="kuadrant-system/authorino-58cd955675-vvvsl" Jul 23 10:02:18.519616 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.519571 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rchvp\" (UniqueName: \"kubernetes.io/projected/c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75-kube-api-access-rchvp\") pod \"authorino-58cd955675-vvvsl\" (UID: \"c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75\") " pod="kuadrant-system/authorino-58cd955675-vvvsl" Jul 23 10:02:18.648021 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.647985 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vvvsl" Jul 23 10:02:18.820658 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:18.820632 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vvvsl"] Jul 23 10:02:18.822723 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:02:18.822700 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6593d19_d9c7_4a8d_9ec4_e6ed75dc6e75.slice/crio-34d002cd91ec1a0b33be01ccee3cd5cf06b7a9afb480717f5f3b48ca4c6ff6b8 WatchSource:0}: Error finding container 34d002cd91ec1a0b33be01ccee3cd5cf06b7a9afb480717f5f3b48ca4c6ff6b8: Status 404 returned error can't find the container with id 34d002cd91ec1a0b33be01ccee3cd5cf06b7a9afb480717f5f3b48ca4c6ff6b8 Jul 23 10:02:19.502549 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:19.502483 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vvvsl" event={"ID":"c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75","Type":"ContainerStarted","Data":"34d002cd91ec1a0b33be01ccee3cd5cf06b7a9afb480717f5f3b48ca4c6ff6b8"} Jul 23 10:02:23.747266 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:23.747150 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vvvsl"] Jul 23 10:02:24.533951 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:24.533909 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vvvsl" event={"ID":"c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75","Type":"ContainerStarted","Data":"338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db"} Jul 23 10:02:24.535431 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:24.535404 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" event={"ID":"e30101a3-6590-47c8-977f-ba5d8503f742","Type":"ContainerStarted","Data":"6e72d318a222d66425707858bd1e20ac2c8015849f01b17b02d5b0f14ecb95bf"} Jul 23 10:02:24.535571 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:24.535459 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" Jul 23 10:02:24.548365 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:24.548306 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-vvvsl" podStartSLOduration=1.899878396 podStartE2EDuration="6.548288294s" totalImagesPullingTime="4.648409898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:02:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:02:24.547165645 +0000 UTC m=+464.794121922" watchObservedRunningTime="2026-07-23 10:02:24.548288294 +0000 UTC m=+464.795244603" Jul 23 10:02:24.563330 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:24.563270 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" podStartSLOduration=2.443343067 podStartE2EDuration="7.563252005s" totalImagesPullingTime="5.119908938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:02:24.56152532 +0000 UTC m=+464.808481588" watchObservedRunningTime="2026-07-23 10:02:24.563252005 +0000 UTC m=+464.810208270" Jul 23 10:02:25.540662 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:25.540618 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-vvvsl" podUID="c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75" containerName="authorino" containerID="cri-o://338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db" gracePeriod=30 Jul 23 10:02:25.796398 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:25.796305 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vvvsl" Jul 23 10:02:25.878523 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:25.878486 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rchvp\" (UniqueName: \"kubernetes.io/projected/c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75-kube-api-access-rchvp\") pod \"c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75\" (UID: \"c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75\") " Jul 23 10:02:25.880680 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:25.880653 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75-kube-api-access-rchvp" (OuterVolumeSpecName: "kube-api-access-rchvp") pod "c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75" (UID: "c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75"). InnerVolumeSpecName "kube-api-access-rchvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:02:25.979330 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:25.979273 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rchvp\" (UniqueName: \"kubernetes.io/projected/c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75-kube-api-access-rchvp\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:02:26.546612 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:26.546516 2580 generic.go:356] "Generic (PLEG): container finished" podID="c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75" containerID="338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db" exitCode=0 Jul 23 10:02:26.546612 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:26.546567 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vvvsl" Jul 23 10:02:26.547041 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:26.546599 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vvvsl" event={"ID":"c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75","Type":"ContainerDied","Data":"338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db"} Jul 23 10:02:26.547041 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:26.546651 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vvvsl" event={"ID":"c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75","Type":"ContainerDied","Data":"34d002cd91ec1a0b33be01ccee3cd5cf06b7a9afb480717f5f3b48ca4c6ff6b8"} Jul 23 10:02:26.547041 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:26.546672 2580 scope.go:117] "RemoveContainer" containerID="338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db" Jul 23 10:02:26.555939 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:26.555912 2580 scope.go:117] "RemoveContainer" containerID="338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db" Jul 23 10:02:26.556259 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:02:26.556239 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db\": container with ID starting with 338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db not found: ID does not exist" containerID="338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db" Jul 23 10:02:26.556333 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:26.556272 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db"} err="failed to get container status \"338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db\": rpc error: code = NotFound desc = could not find container \"338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db\": container with ID starting with 338cd3f7d4b6af4f0f96a6374214d69a91083c87b26394e2c4fe1f122a08f2db not found: ID does not exist" Jul 23 10:02:26.562454 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:26.562425 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vvvsl"] Jul 23 10:02:26.564272 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:26.564248 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-vvvsl"] Jul 23 10:02:28.280465 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:28.280434 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75" path="/var/lib/kubelet/pods/c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75/volumes" Jul 23 10:02:35.541740 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:35.541703 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b5268" Jul 23 10:02:51.404945 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.404916 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wqb6d"] Jul 23 10:02:51.405403 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.405239 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75" containerName="authorino" Jul 23 10:02:51.405403 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.405250 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75" containerName="authorino" Jul 23 10:02:51.405403 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.405318 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6593d19-d9c7-4a8d-9ec4-e6ed75dc6e75" containerName="authorino" Jul 23 10:02:51.409317 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.409299 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wqb6d" Jul 23 10:02:51.411483 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.411461 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-f6qjl\"" Jul 23 10:02:51.413106 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.413071 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wqb6d"] Jul 23 10:02:51.496097 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.496060 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q2sd\" (UniqueName: \"kubernetes.io/projected/68cab8c7-c600-4672-8fc4-ca994ab624a5-kube-api-access-5q2sd\") pod \"authorino-5d4b67c446-wqb6d\" (UID: \"68cab8c7-c600-4672-8fc4-ca994ab624a5\") " pod="kuadrant-system/authorino-5d4b67c446-wqb6d" Jul 23 10:02:51.596493 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.596450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q2sd\" (UniqueName: \"kubernetes.io/projected/68cab8c7-c600-4672-8fc4-ca994ab624a5-kube-api-access-5q2sd\") pod \"authorino-5d4b67c446-wqb6d\" (UID: \"68cab8c7-c600-4672-8fc4-ca994ab624a5\") " pod="kuadrant-system/authorino-5d4b67c446-wqb6d" Jul 23 10:02:51.604323 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.604297 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q2sd\" (UniqueName: \"kubernetes.io/projected/68cab8c7-c600-4672-8fc4-ca994ab624a5-kube-api-access-5q2sd\") pod \"authorino-5d4b67c446-wqb6d\" (UID: \"68cab8c7-c600-4672-8fc4-ca994ab624a5\") " pod="kuadrant-system/authorino-5d4b67c446-wqb6d" Jul 23 10:02:51.652235 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.652201 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wqb6d"] Jul 23 10:02:51.652472 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.652460 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wqb6d" Jul 23 10:02:51.787946 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:51.787911 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wqb6d"] Jul 23 10:02:51.791383 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:02:51.791351 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68cab8c7_c600_4672_8fc4_ca994ab624a5.slice/crio-e065ce10d76accf20feaef20c77618f8077a215d5319e0c1e289997b26aadda6 WatchSource:0}: Error finding container e065ce10d76accf20feaef20c77618f8077a215d5319e0c1e289997b26aadda6: Status 404 returned error can't find the container with id e065ce10d76accf20feaef20c77618f8077a215d5319e0c1e289997b26aadda6 Jul 23 10:02:52.677241 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:52.677074 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-wqb6d" podUID="68cab8c7-c600-4672-8fc4-ca994ab624a5" containerName="authorino" containerID="cri-o://cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213" gracePeriod=30 Jul 23 10:02:52.677241 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:52.677089 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wqb6d" event={"ID":"68cab8c7-c600-4672-8fc4-ca994ab624a5","Type":"ContainerStarted","Data":"cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213"} Jul 23 10:02:52.678018 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:52.677270 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wqb6d" event={"ID":"68cab8c7-c600-4672-8fc4-ca994ab624a5","Type":"ContainerStarted","Data":"e065ce10d76accf20feaef20c77618f8077a215d5319e0c1e289997b26aadda6"} Jul 23 10:02:52.690447 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:52.690391 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-wqb6d" podStartSLOduration=0.932197678 podStartE2EDuration="1.690370215s" totalImagesPullingTime="758.172537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:02:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:02:52.689289905 +0000 UTC m=+492.936246170" watchObservedRunningTime="2026-07-23 10:02:52.690370215 +0000 UTC m=+492.937326477" Jul 23 10:02:52.934827 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:52.934792 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wqb6d" Jul 23 10:02:53.009816 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.009779 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q2sd\" (UniqueName: \"kubernetes.io/projected/68cab8c7-c600-4672-8fc4-ca994ab624a5-kube-api-access-5q2sd\") pod \"68cab8c7-c600-4672-8fc4-ca994ab624a5\" (UID: \"68cab8c7-c600-4672-8fc4-ca994ab624a5\") " Jul 23 10:02:53.011987 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.011952 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68cab8c7-c600-4672-8fc4-ca994ab624a5-kube-api-access-5q2sd" (OuterVolumeSpecName: "kube-api-access-5q2sd") pod "68cab8c7-c600-4672-8fc4-ca994ab624a5" (UID: "68cab8c7-c600-4672-8fc4-ca994ab624a5"). InnerVolumeSpecName "kube-api-access-5q2sd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:02:53.110564 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.110457 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q2sd\" (UniqueName: \"kubernetes.io/projected/68cab8c7-c600-4672-8fc4-ca994ab624a5-kube-api-access-5q2sd\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:02:53.683153 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.683112 2580 generic.go:356] "Generic (PLEG): container finished" podID="68cab8c7-c600-4672-8fc4-ca994ab624a5" containerID="cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213" exitCode=0 Jul 23 10:02:53.683153 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.683159 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wqb6d" Jul 23 10:02:53.683652 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.683228 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wqb6d" event={"ID":"68cab8c7-c600-4672-8fc4-ca994ab624a5","Type":"ContainerDied","Data":"cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213"} Jul 23 10:02:53.683652 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.683271 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-wqb6d" event={"ID":"68cab8c7-c600-4672-8fc4-ca994ab624a5","Type":"ContainerDied","Data":"e065ce10d76accf20feaef20c77618f8077a215d5319e0c1e289997b26aadda6"} Jul 23 10:02:53.683652 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.683290 2580 scope.go:117] "RemoveContainer" containerID="cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213" Jul 23 10:02:53.692954 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.692936 2580 scope.go:117] "RemoveContainer" containerID="cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213" Jul 23 10:02:53.693277 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:02:53.693258 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213\": container with ID starting with cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213 not found: ID does not exist" containerID="cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213" Jul 23 10:02:53.693357 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.693286 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213"} err="failed to get container status \"cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213\": rpc error: code = NotFound desc = could not find container \"cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213\": container with ID starting with cdf17d2eb01749f4dc18190ce98ce1e2a006b53806cb00630ac6800a54717213 not found: ID does not exist" Jul 23 10:02:53.701875 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.701846 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wqb6d"] Jul 23 10:02:53.705276 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:53.705251 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wqb6d"] Jul 23 10:02:54.280142 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:02:54.280109 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68cab8c7-c600-4672-8fc4-ca994ab624a5" path="/var/lib/kubelet/pods/68cab8c7-c600-4672-8fc4-ca994ab624a5/volumes" Jul 23 10:03:35.297914 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:03:35.297830 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:04:11.175150 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:11.175112 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:04:17.580046 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:17.579999 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:04:27.680001 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.679966 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk"] Jul 23 10:04:27.680462 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.680431 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68cab8c7-c600-4672-8fc4-ca994ab624a5" containerName="authorino" Jul 23 10:04:27.680462 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.680450 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="68cab8c7-c600-4672-8fc4-ca994ab624a5" containerName="authorino" Jul 23 10:04:27.680568 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.680525 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="68cab8c7-c600-4672-8fc4-ca994ab624a5" containerName="authorino" Jul 23 10:04:27.683763 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.683747 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.685705 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.685685 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 10:04:27.685912 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.685899 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 10:04:27.686606 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.686589 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 23 10:04:27.686668 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.686636 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qxbzn\"" Jul 23 10:04:27.692161 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.692134 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk"] Jul 23 10:04:27.774914 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.774872 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.774914 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.774911 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.775181 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.774934 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.775181 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.774949 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08d2ea1d-1238-417f-bc32-a59a50645324-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.775181 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.775021 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j47t\" (UniqueName: \"kubernetes.io/projected/08d2ea1d-1238-417f-bc32-a59a50645324-kube-api-access-6j47t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.775181 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.775070 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.876415 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.876379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.876415 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.876419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.876638 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.876444 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.876638 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.876459 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08d2ea1d-1238-417f-bc32-a59a50645324-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.876638 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.876477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j47t\" (UniqueName: \"kubernetes.io/projected/08d2ea1d-1238-417f-bc32-a59a50645324-kube-api-access-6j47t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.876638 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.876505 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.876897 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.876870 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.876954 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.876903 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.876954 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.876928 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.878914 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.878895 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08d2ea1d-1238-417f-bc32-a59a50645324-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.879208 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.879169 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08d2ea1d-1238-417f-bc32-a59a50645324-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.884136 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.884108 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j47t\" (UniqueName: \"kubernetes.io/projected/08d2ea1d-1238-417f-bc32-a59a50645324-kube-api-access-6j47t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bnvgk\" (UID: \"08d2ea1d-1238-417f-bc32-a59a50645324\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:27.995385 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:27.995332 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:28.130243 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:28.130219 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk"] Jul 23 10:04:28.132352 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:04:28.132321 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d2ea1d_1238_417f_bc32_a59a50645324.slice/crio-5d0aa45749e48a7c3966dc1a8d0cc867d3d7b0511eb484a97078c4d7437ba527 WatchSource:0}: Error finding container 5d0aa45749e48a7c3966dc1a8d0cc867d3d7b0511eb484a97078c4d7437ba527: Status 404 returned error can't find the container with id 5d0aa45749e48a7c3966dc1a8d0cc867d3d7b0511eb484a97078c4d7437ba527 Jul 23 10:04:28.165828 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:28.165796 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" event={"ID":"08d2ea1d-1238-417f-bc32-a59a50645324","Type":"ContainerStarted","Data":"5d0aa45749e48a7c3966dc1a8d0cc867d3d7b0511eb484a97078c4d7437ba527"} Jul 23 10:04:28.885429 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:28.885362 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:04:30.178499 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:30.178462 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" event={"ID":"08d2ea1d-1238-417f-bc32-a59a50645324","Type":"ContainerStarted","Data":"8490c481c88b3218cd82cb6760a730e259a3dbc67a6180c0eda1749b980ee8cf"} Jul 23 10:04:30.194133 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:30.194085 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" podStartSLOduration=2.046046958 podStartE2EDuration="3.194067036s" totalImagesPullingTime="1.148020078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:04:30.193485295 +0000 UTC m=+590.440441561" watchObservedRunningTime="2026-07-23 10:04:30.194067036 +0000 UTC m=+590.441023302" Jul 23 10:04:37.996483 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:37.996422 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:37.996941 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:37.996517 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:38.009375 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:38.009347 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:38.232348 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:38.232319 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bnvgk" Jul 23 10:04:40.192536 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:40.192510 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:04:40.193660 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:40.193634 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:04:41.175713 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:04:41.175673 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:05:09.179770 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:09.179682 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:05:24.474883 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:24.474846 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:05:53.473132 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.473095 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6466556dd9-jxx7j"] Jul 23 10:05:53.475543 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.475522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6466556dd9-jxx7j" Jul 23 10:05:53.477993 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.477972 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-f6qjl\"" Jul 23 10:05:53.478049 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.477972 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 23 10:05:53.484420 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.484394 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6466556dd9-jxx7j"] Jul 23 10:05:53.629272 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.629239 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/739ec57c-e508-4d7b-b18a-8f3b7ea49b92-tls-cert\") pod \"authorino-6466556dd9-jxx7j\" (UID: \"739ec57c-e508-4d7b-b18a-8f3b7ea49b92\") " pod="kuadrant-system/authorino-6466556dd9-jxx7j" Jul 23 10:05:53.629458 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.629327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z88f8\" (UniqueName: \"kubernetes.io/projected/739ec57c-e508-4d7b-b18a-8f3b7ea49b92-kube-api-access-z88f8\") pod \"authorino-6466556dd9-jxx7j\" (UID: \"739ec57c-e508-4d7b-b18a-8f3b7ea49b92\") " pod="kuadrant-system/authorino-6466556dd9-jxx7j" Jul 23 10:05:53.730705 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.730619 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z88f8\" (UniqueName: \"kubernetes.io/projected/739ec57c-e508-4d7b-b18a-8f3b7ea49b92-kube-api-access-z88f8\") pod \"authorino-6466556dd9-jxx7j\" (UID: \"739ec57c-e508-4d7b-b18a-8f3b7ea49b92\") " pod="kuadrant-system/authorino-6466556dd9-jxx7j" Jul 23 10:05:53.730705 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.730677 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/739ec57c-e508-4d7b-b18a-8f3b7ea49b92-tls-cert\") pod \"authorino-6466556dd9-jxx7j\" (UID: \"739ec57c-e508-4d7b-b18a-8f3b7ea49b92\") " pod="kuadrant-system/authorino-6466556dd9-jxx7j" Jul 23 10:05:53.733184 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.733161 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/739ec57c-e508-4d7b-b18a-8f3b7ea49b92-tls-cert\") pod \"authorino-6466556dd9-jxx7j\" (UID: \"739ec57c-e508-4d7b-b18a-8f3b7ea49b92\") " pod="kuadrant-system/authorino-6466556dd9-jxx7j" Jul 23 10:05:53.738294 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.738265 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z88f8\" (UniqueName: \"kubernetes.io/projected/739ec57c-e508-4d7b-b18a-8f3b7ea49b92-kube-api-access-z88f8\") pod \"authorino-6466556dd9-jxx7j\" (UID: \"739ec57c-e508-4d7b-b18a-8f3b7ea49b92\") " pod="kuadrant-system/authorino-6466556dd9-jxx7j" Jul 23 10:05:53.785682 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.785645 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6466556dd9-jxx7j" Jul 23 10:05:53.917100 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.917073 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6466556dd9-jxx7j"] Jul 23 10:05:53.919686 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:05:53.919659 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod739ec57c_e508_4d7b_b18a_8f3b7ea49b92.slice/crio-5cf40a8eddea9397665573fcbf6192e7caa8672d47c5baf6f1e2cab81408db0b WatchSource:0}: Error finding container 5cf40a8eddea9397665573fcbf6192e7caa8672d47c5baf6f1e2cab81408db0b: Status 404 returned error can't find the container with id 5cf40a8eddea9397665573fcbf6192e7caa8672d47c5baf6f1e2cab81408db0b Jul 23 10:05:53.921368 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:53.921344 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 10:05:54.608541 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:54.608506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6466556dd9-jxx7j" event={"ID":"739ec57c-e508-4d7b-b18a-8f3b7ea49b92","Type":"ContainerStarted","Data":"cf761c0469ab1abbf72eea0b2984c0f748e6b47bf548a011a42bb6a784cfd352"} Jul 23 10:05:54.608541 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:54.608548 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6466556dd9-jxx7j" event={"ID":"739ec57c-e508-4d7b-b18a-8f3b7ea49b92","Type":"ContainerStarted","Data":"5cf40a8eddea9397665573fcbf6192e7caa8672d47c5baf6f1e2cab81408db0b"} Jul 23 10:05:54.624037 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:05:54.623987 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6466556dd9-jxx7j" podStartSLOduration=1.160892377 podStartE2EDuration="1.623968452s" totalImagesPullingTime="463.076075ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:05:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:05:54.622996116 +0000 UTC m=+674.869952382" watchObservedRunningTime="2026-07-23 10:05:54.623968452 +0000 UTC m=+674.870924717" Jul 23 10:06:26.981008 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:06:26.980970 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:06:36.786500 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:06:36.786412 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:06:47.080005 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:06:47.079958 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:08:30.114356 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:08:30.114320 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:08:38.792026 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:08:38.791991 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:09:11.284849 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:09:11.284809 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:09:20.773669 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:09:20.773631 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:09:29.184608 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:09:29.184570 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:09:38.280530 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:09:38.280450 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:09:40.226071 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:09:40.226043 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:09:40.226546 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:09:40.226109 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:09:48.681678 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:09:48.681640 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:09:59.178855 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:09:59.178817 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:10:09.978828 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:10:09.978786 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:10:20.178724 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:10:20.178692 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:10:42.180608 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:10:42.180577 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:10:53.583245 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:10:53.583204 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:11:17.875426 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:11:17.875330 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:11:35.880033 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:11:35.879994 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:11:47.977034 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:11:47.976990 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:11:57.278786 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:11:57.278755 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:12:06.678108 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:12:06.678061 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:12:16.281222 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:12:16.281175 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:12:27.573284 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:12:27.573244 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:14:14.608864 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:14:14.608829 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:14:23.772898 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:14:23.772863 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:14:36.280387 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:14:36.280356 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:14:40.260157 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:14:40.260132 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:14:40.260850 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:14:40.260827 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:14:48.580884 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:14:48.580848 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:14:58.201331 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:14:58.201294 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:15:09.184616 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:15:09.184580 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:15:16.296357 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:15:16.296313 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:15:25.478367 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:15:25.478330 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:15:34.383352 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:15:34.383274 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:15:43.979676 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:15:43.979632 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:15:55.799155 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:15:55.799121 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:16:03.382722 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:16:03.382686 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:16:20.985367 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:16:20.985333 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:16:31.172421 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:16:31.172372 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:16:49.486939 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:16:49.486907 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:17:06.280673 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:17:06.280592 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:17:23.389051 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:17:23.389017 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:17:36.777440 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:17:36.777402 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:17:43.176121 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:17:43.176082 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:17:59.585810 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:17:59.585775 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:18:09.278037 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:18:09.278005 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:18:18.673593 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:18:18.673559 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:18:26.281683 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:18:26.281653 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:18:35.883069 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:18:35.882982 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:18:44.780998 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:18:44.780957 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:18:59.178057 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:18:59.178022 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:19:07.883822 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:19:07.883784 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:19:17.580395 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:19:17.580355 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:19:29.589508 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:19:29.589468 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:19:38.771645 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:19:38.771608 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:19:40.300042 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:19:40.300012 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:19:40.301293 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:19:40.301273 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:19:46.987962 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:19:46.987928 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:19:57.686770 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:19:57.686729 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:20:09.876820 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:20:09.876737 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:20:20.873662 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:20:20.873626 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:20:29.675185 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:20:29.675149 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:20:52.088527 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:20:52.088489 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:21:05.589220 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:21:05.589168 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:21:39.079042 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:21:39.078964 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:21:49.776135 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:21:49.776100 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:22:22.776403 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:22:22.776364 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:22:33.672617 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:22:33.672578 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:23:06.990020 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:23:06.989937 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:23:18.475132 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:23:18.475094 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:23:51.283959 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:23:51.283923 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:24:40.334346 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:24:40.334315 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:24:40.336761 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:24:40.336739 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:25:24.681507 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:25:24.681471 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:25:34.279724 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:25:34.279690 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:25:45.178477 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:25:45.178439 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:26:04.181696 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:26:04.181615 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:26:42.593589 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:26:42.593554 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:26:54.191171 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:26:54.191132 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:27:09.518896 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:27:09.518862 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:27:27.501473 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:27:27.501442 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:27:36.105498 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:27:36.105413 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:28:27.988966 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:28:27.988930 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:28:37.793761 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:28:37.793714 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:29:29.381086 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:29.381041 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:29:40.365518 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:40.365490 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:29:40.375638 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:40.375613 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:29:42.081604 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:42.081562 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:29:43.563728 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.563697 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6"] Jul 23 10:29:43.566957 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.566933 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.569777 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.569751 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d95f36\"/\"openshift-service-ca.crt\"" Jul 23 10:29:43.569777 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.569753 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d95f36\"/\"test-model-d95f36-kserve-self-signed-certs\"" Jul 23 10:29:43.569944 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.569874 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d95f36\"/\"default-dockercfg-rr86n\"" Jul 23 10:29:43.569985 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.569966 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d95f36\"/\"kube-root-ca.crt\"" Jul 23 10:29:43.578007 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.577981 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6"] Jul 23 10:29:43.720895 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.720855 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzh9h\" (UniqueName: \"kubernetes.io/projected/d43cca78-5258-4d21-889c-47221500b255-kube-api-access-gzh9h\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.720895 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.720904 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-tmp-dir\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.721127 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.720940 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-dshm\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.721127 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.720988 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-model-cache\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.721127 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.721018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-home\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.721127 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.721044 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43cca78-5258-4d21-889c-47221500b255-tls-certs\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.821935 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.821843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-model-cache\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.821935 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.821888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-home\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.821935 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.821914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43cca78-5258-4d21-889c-47221500b255-tls-certs\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.822233 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.821950 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzh9h\" (UniqueName: \"kubernetes.io/projected/d43cca78-5258-4d21-889c-47221500b255-kube-api-access-gzh9h\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.822233 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.821988 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-tmp-dir\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.822233 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.822144 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-dshm\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.822401 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.822385 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-model-cache\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.822462 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.822443 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-tmp-dir\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.822498 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.822461 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-home\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.824461 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.824438 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-dshm\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.824632 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.824615 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43cca78-5258-4d21-889c-47221500b255-tls-certs\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.830378 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.830359 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzh9h\" (UniqueName: \"kubernetes.io/projected/d43cca78-5258-4d21-889c-47221500b255-kube-api-access-gzh9h\") pod \"test-model-d95f36-kserve-7c75f4c4cf-9mpz6\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:43.875723 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:43.875678 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:44.008530 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:44.008502 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6"] Jul 23 10:29:44.010959 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:29:44.010926 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd43cca78_5258_4d21_889c_47221500b255.slice/crio-4217f3e5719c0d64fff49fdc5feef3e5c2b9377f85a2772a348f60c7d0efb4e7 WatchSource:0}: Error finding container 4217f3e5719c0d64fff49fdc5feef3e5c2b9377f85a2772a348f60c7d0efb4e7: Status 404 returned error can't find the container with id 4217f3e5719c0d64fff49fdc5feef3e5c2b9377f85a2772a348f60c7d0efb4e7 Jul 23 10:29:44.767630 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:44.767594 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" event={"ID":"d43cca78-5258-4d21-889c-47221500b255","Type":"ContainerStarted","Data":"52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f"} Jul 23 10:29:44.767630 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:44.767638 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" event={"ID":"d43cca78-5258-4d21-889c-47221500b255","Type":"ContainerStarted","Data":"4217f3e5719c0d64fff49fdc5feef3e5c2b9377f85a2772a348f60c7d0efb4e7"} Jul 23 10:29:44.783956 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:44.783903 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" podStartSLOduration=1.783885631 podStartE2EDuration="1.783885631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:29:44.781676953 +0000 UTC m=+2105.028633218" watchObservedRunningTime="2026-07-23 10:29:44.783885631 +0000 UTC m=+2105.030841896" Jul 23 10:29:51.881883 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:51.881843 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:29:53.876284 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:53.876241 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:53.876640 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:53.876301 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:53.888328 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:53.888305 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:29:54.829485 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:29:54.829453 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:30:00.152696 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:00.152666 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746710-lhmmp"] Jul 23 10:30:00.156127 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:00.156110 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746710-lhmmp" Jul 23 10:30:00.159756 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:00.159735 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2vbpd\"" Jul 23 10:30:00.167000 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:00.166979 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746710-lhmmp" Jul 23 10:30:00.168135 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:00.168112 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746710-lhmmp"] Jul 23 10:30:00.307090 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:00.307067 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746710-lhmmp"] Jul 23 10:30:00.309514 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:30:00.309488 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fd7f04c_5055_43bb_b204_301bbe9fdbb6.slice/crio-22aaf89582a7750d74b472edfd5b028f08aa1fe5d397951a095c7c6930d3457f WatchSource:0}: Error finding container 22aaf89582a7750d74b472edfd5b028f08aa1fe5d397951a095c7c6930d3457f: Status 404 returned error can't find the container with id 22aaf89582a7750d74b472edfd5b028f08aa1fe5d397951a095c7c6930d3457f Jul 23 10:30:00.311226 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:00.311205 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 10:30:00.848117 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:00.848076 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746710-lhmmp" event={"ID":"7fd7f04c-5055-43bb-b204-301bbe9fdbb6","Type":"ContainerStarted","Data":"22aaf89582a7750d74b472edfd5b028f08aa1fe5d397951a095c7c6930d3457f"} Jul 23 10:30:03.866266 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:03.866232 2580 generic.go:356] "Generic (PLEG): container finished" podID="7fd7f04c-5055-43bb-b204-301bbe9fdbb6" containerID="4d4914d0f2ac9c36c0d94f7e0cb5504f47d4375b768510382f2ebd0b8c410d42" exitCode=0 Jul 23 10:30:03.866654 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:03.866308 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746710-lhmmp" event={"ID":"7fd7f04c-5055-43bb-b204-301bbe9fdbb6","Type":"ContainerDied","Data":"4d4914d0f2ac9c36c0d94f7e0cb5504f47d4375b768510382f2ebd0b8c410d42"} Jul 23 10:30:05.001487 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:05.001460 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746710-lhmmp" Jul 23 10:30:05.876820 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:05.876787 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746710-lhmmp" Jul 23 10:30:05.876999 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:05.876815 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746710-lhmmp" event={"ID":"7fd7f04c-5055-43bb-b204-301bbe9fdbb6","Type":"ContainerDied","Data":"22aaf89582a7750d74b472edfd5b028f08aa1fe5d397951a095c7c6930d3457f"} Jul 23 10:30:05.876999 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:05.876866 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22aaf89582a7750d74b472edfd5b028f08aa1fe5d397951a095c7c6930d3457f" Jul 23 10:30:24.677909 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:24.677871 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:30:38.490045 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:30:38.489963 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:31:01.286045 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:01.286012 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6"] Jul 23 10:31:02.420052 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.420017 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q"] Jul 23 10:31:02.420456 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.420367 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fd7f04c-5055-43bb-b204-301bbe9fdbb6" containerName="cleanup" Jul 23 10:31:02.420456 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.420378 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fd7f04c-5055-43bb-b204-301bbe9fdbb6" containerName="cleanup" Jul 23 10:31:02.420456 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.420441 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fd7f04c-5055-43bb-b204-301bbe9fdbb6" containerName="cleanup" Jul 23 10:31:02.423446 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.423430 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" Jul 23 10:31:02.425706 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.425687 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2vbpd\"" Jul 23 10:31:02.425794 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.425691 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 10:31:02.435881 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.435858 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q"] Jul 23 10:31:02.503442 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.503406 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f798c7c-7b94-4ed7-8e8e-0e575809bc40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q\" (UID: \"0f798c7c-7b94-4ed7-8e8e-0e575809bc40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" Jul 23 10:31:02.604426 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.604388 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f798c7c-7b94-4ed7-8e8e-0e575809bc40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q\" (UID: \"0f798c7c-7b94-4ed7-8e8e-0e575809bc40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" Jul 23 10:31:02.605044 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.605025 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f798c7c-7b94-4ed7-8e8e-0e575809bc40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q\" (UID: \"0f798c7c-7b94-4ed7-8e8e-0e575809bc40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" Jul 23 10:31:02.733912 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.733876 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" Jul 23 10:31:02.870944 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:02.870917 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q"] Jul 23 10:31:02.872994 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:31:02.872969 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f798c7c_7b94_4ed7_8e8e_0e575809bc40.slice/crio-390d9f9a848c4fac12fe70617bad08ac4210871830bd20aaa74dea124fd245b0 WatchSource:0}: Error finding container 390d9f9a848c4fac12fe70617bad08ac4210871830bd20aaa74dea124fd245b0: Status 404 returned error can't find the container with id 390d9f9a848c4fac12fe70617bad08ac4210871830bd20aaa74dea124fd245b0 Jul 23 10:31:03.156654 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:03.156621 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" event={"ID":"0f798c7c-7b94-4ed7-8e8e-0e575809bc40","Type":"ContainerStarted","Data":"4814ae39f142ff3d460c3064ae7d4ff9ed9e3d5ac8152d623e53f6313e8fe224"} Jul 23 10:31:03.156825 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:03.156669 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" event={"ID":"0f798c7c-7b94-4ed7-8e8e-0e575809bc40","Type":"ContainerStarted","Data":"390d9f9a848c4fac12fe70617bad08ac4210871830bd20aaa74dea124fd245b0"} Jul 23 10:31:03.172737 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:03.172687 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" podStartSLOduration=1.172671289 podStartE2EDuration="1.172671289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:31:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:31:03.171012095 +0000 UTC m=+2183.417968372" watchObservedRunningTime="2026-07-23 10:31:03.172671289 +0000 UTC m=+2183.419627553" Jul 23 10:31:04.164007 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:04.163973 2580 generic.go:356] "Generic (PLEG): container finished" podID="0f798c7c-7b94-4ed7-8e8e-0e575809bc40" containerID="4814ae39f142ff3d460c3064ae7d4ff9ed9e3d5ac8152d623e53f6313e8fe224" exitCode=0 Jul 23 10:31:04.164439 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:04.164063 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" event={"ID":"0f798c7c-7b94-4ed7-8e8e-0e575809bc40","Type":"ContainerDied","Data":"4814ae39f142ff3d460c3064ae7d4ff9ed9e3d5ac8152d623e53f6313e8fe224"} Jul 23 10:31:05.304633 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:05.304604 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" Jul 23 10:31:05.429097 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:05.429018 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f798c7c-7b94-4ed7-8e8e-0e575809bc40-maas-api-service-ca\") pod \"0f798c7c-7b94-4ed7-8e8e-0e575809bc40\" (UID: \"0f798c7c-7b94-4ed7-8e8e-0e575809bc40\") " Jul 23 10:31:05.429393 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:05.429369 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f798c7c-7b94-4ed7-8e8e-0e575809bc40-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0f798c7c-7b94-4ed7-8e8e-0e575809bc40" (UID: "0f798c7c-7b94-4ed7-8e8e-0e575809bc40"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 10:31:05.530301 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:05.530265 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f798c7c-7b94-4ed7-8e8e-0e575809bc40-maas-api-service-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:31:06.174988 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:06.174954 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" Jul 23 10:31:06.175163 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:06.174987 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q" event={"ID":"0f798c7c-7b94-4ed7-8e8e-0e575809bc40","Type":"ContainerDied","Data":"390d9f9a848c4fac12fe70617bad08ac4210871830bd20aaa74dea124fd245b0"} Jul 23 10:31:06.175163 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:06.175065 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="390d9f9a848c4fac12fe70617bad08ac4210871830bd20aaa74dea124fd245b0" Jul 23 10:31:06.706151 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:06.706115 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:31:16.305870 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.305831 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" podUID="d43cca78-5258-4d21-889c-47221500b255" containerName="main" containerID="cri-o://52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f" gracePeriod=45 Jul 23 10:31:16.554260 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.554235 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:31:16.733915 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.733881 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-dshm\") pod \"d43cca78-5258-4d21-889c-47221500b255\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " Jul 23 10:31:16.734079 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.733923 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-model-cache\") pod \"d43cca78-5258-4d21-889c-47221500b255\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " Jul 23 10:31:16.734079 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.733974 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzh9h\" (UniqueName: \"kubernetes.io/projected/d43cca78-5258-4d21-889c-47221500b255-kube-api-access-gzh9h\") pod \"d43cca78-5258-4d21-889c-47221500b255\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " Jul 23 10:31:16.734079 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.733989 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-tmp-dir\") pod \"d43cca78-5258-4d21-889c-47221500b255\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " Jul 23 10:31:16.734079 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.734026 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43cca78-5258-4d21-889c-47221500b255-tls-certs\") pod \"d43cca78-5258-4d21-889c-47221500b255\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " Jul 23 10:31:16.734293 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.734223 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-home\") pod \"d43cca78-5258-4d21-889c-47221500b255\" (UID: \"d43cca78-5258-4d21-889c-47221500b255\") " Jul 23 10:31:16.734350 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.734296 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-model-cache" (OuterVolumeSpecName: "model-cache") pod "d43cca78-5258-4d21-889c-47221500b255" (UID: "d43cca78-5258-4d21-889c-47221500b255"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:31:16.734350 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.734311 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d43cca78-5258-4d21-889c-47221500b255" (UID: "d43cca78-5258-4d21-889c-47221500b255"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:31:16.734478 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.734450 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-home" (OuterVolumeSpecName: "home") pod "d43cca78-5258-4d21-889c-47221500b255" (UID: "d43cca78-5258-4d21-889c-47221500b255"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:31:16.734579 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.734564 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-home\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:31:16.734629 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.734584 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-model-cache\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:31:16.734629 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.734600 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-tmp-dir\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:31:16.736363 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.736343 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-dshm" (OuterVolumeSpecName: "dshm") pod "d43cca78-5258-4d21-889c-47221500b255" (UID: "d43cca78-5258-4d21-889c-47221500b255"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:31:16.736453 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.736434 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d43cca78-5258-4d21-889c-47221500b255-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d43cca78-5258-4d21-889c-47221500b255" (UID: "d43cca78-5258-4d21-889c-47221500b255"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 10:31:16.736489 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.736470 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d43cca78-5258-4d21-889c-47221500b255-kube-api-access-gzh9h" (OuterVolumeSpecName: "kube-api-access-gzh9h") pod "d43cca78-5258-4d21-889c-47221500b255" (UID: "d43cca78-5258-4d21-889c-47221500b255"). InnerVolumeSpecName "kube-api-access-gzh9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:31:16.835008 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.834971 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d43cca78-5258-4d21-889c-47221500b255-dshm\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:31:16.835008 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.835001 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzh9h\" (UniqueName: \"kubernetes.io/projected/d43cca78-5258-4d21-889c-47221500b255-kube-api-access-gzh9h\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:31:16.835008 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:16.835013 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43cca78-5258-4d21-889c-47221500b255-tls-certs\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:31:17.232541 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:17.232503 2580 generic.go:356] "Generic (PLEG): container finished" podID="d43cca78-5258-4d21-889c-47221500b255" containerID="52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f" exitCode=0 Jul 23 10:31:17.232749 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:17.232571 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" Jul 23 10:31:17.232749 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:17.232595 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" event={"ID":"d43cca78-5258-4d21-889c-47221500b255","Type":"ContainerDied","Data":"52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f"} Jul 23 10:31:17.232749 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:17.232639 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6" event={"ID":"d43cca78-5258-4d21-889c-47221500b255","Type":"ContainerDied","Data":"4217f3e5719c0d64fff49fdc5feef3e5c2b9377f85a2772a348f60c7d0efb4e7"} Jul 23 10:31:17.232749 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:17.232656 2580 scope.go:117] "RemoveContainer" containerID="52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f" Jul 23 10:31:17.242220 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:17.242172 2580 scope.go:117] "RemoveContainer" containerID="52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f" Jul 23 10:31:17.242602 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:31:17.242580 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f\": container with ID starting with 52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f not found: ID does not exist" containerID="52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f" Jul 23 10:31:17.242697 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:17.242611 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f"} err="failed to get container status \"52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f\": rpc error: code = NotFound desc = could not find container \"52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f\": container with ID starting with 52b61fb89918f4bb38dc4aac1223ff448359f26eb5232da78c5e8ff8f86e693f not found: ID does not exist" Jul 23 10:31:17.255718 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:17.255685 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6"] Jul 23 10:31:17.258852 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:17.258829 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d95f36/test-model-d95f36-kserve-7c75f4c4cf-9mpz6"] Jul 23 10:31:17.508548 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:17.508471 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q"] Jul 23 10:31:17.510969 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:17.510947 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d95f36-8xs5q"] Jul 23 10:31:18.279907 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:18.279871 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f798c7c-7b94-4ed7-8e8e-0e575809bc40" path="/var/lib/kubelet/pods/0f798c7c-7b94-4ed7-8e8e-0e575809bc40/volumes" Jul 23 10:31:18.280272 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:18.280253 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d43cca78-5258-4d21-889c-47221500b255" path="/var/lib/kubelet/pods/d43cca78-5258-4d21-889c-47221500b255/volumes" Jul 23 10:31:22.476043 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:22.475999 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:31:24.584142 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.584101 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476"] Jul 23 10:31:24.584539 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.584480 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d43cca78-5258-4d21-889c-47221500b255" containerName="main" Jul 23 10:31:24.584539 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.584492 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d43cca78-5258-4d21-889c-47221500b255" containerName="main" Jul 23 10:31:24.584539 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.584512 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f798c7c-7b94-4ed7-8e8e-0e575809bc40" containerName="revoke-keys" Jul 23 10:31:24.584539 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.584518 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f798c7c-7b94-4ed7-8e8e-0e575809bc40" containerName="revoke-keys" Jul 23 10:31:24.584660 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.584577 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d43cca78-5258-4d21-889c-47221500b255" containerName="main" Jul 23 10:31:24.584660 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.584588 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f798c7c-7b94-4ed7-8e8e-0e575809bc40" containerName="revoke-keys" Jul 23 10:31:24.588795 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.588777 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" Jul 23 10:31:24.590819 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.590801 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 10:31:24.590927 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.590912 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2vbpd\"" Jul 23 10:31:24.595979 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.595956 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476"] Jul 23 10:31:24.704860 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.704821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcea4dbf-546a-4737-8d38-9dfbb12f09a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476\" (UID: \"bcea4dbf-546a-4737-8d38-9dfbb12f09a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" Jul 23 10:31:24.805743 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.805708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcea4dbf-546a-4737-8d38-9dfbb12f09a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476\" (UID: \"bcea4dbf-546a-4737-8d38-9dfbb12f09a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" Jul 23 10:31:24.806328 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.806307 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcea4dbf-546a-4737-8d38-9dfbb12f09a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476\" (UID: \"bcea4dbf-546a-4737-8d38-9dfbb12f09a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" Jul 23 10:31:24.899760 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:24.899655 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" Jul 23 10:31:25.038999 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:25.038972 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476"] Jul 23 10:31:25.040780 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:31:25.040742 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcea4dbf_546a_4737_8d38_9dfbb12f09a7.slice/crio-f77f172ae0a4a7a6621312996ba6774bcb5d4d8bef92831d77ad421fcaf8fceb WatchSource:0}: Error finding container f77f172ae0a4a7a6621312996ba6774bcb5d4d8bef92831d77ad421fcaf8fceb: Status 404 returned error can't find the container with id f77f172ae0a4a7a6621312996ba6774bcb5d4d8bef92831d77ad421fcaf8fceb Jul 23 10:31:25.279565 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:25.279530 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" event={"ID":"bcea4dbf-546a-4737-8d38-9dfbb12f09a7","Type":"ContainerStarted","Data":"0b8ed734dafd240ae8fb638fcf9fe2f71dda803cb91992893467b8c5dbcad62c"} Jul 23 10:31:25.279714 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:25.279592 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" event={"ID":"bcea4dbf-546a-4737-8d38-9dfbb12f09a7","Type":"ContainerStarted","Data":"f77f172ae0a4a7a6621312996ba6774bcb5d4d8bef92831d77ad421fcaf8fceb"} Jul 23 10:31:25.297628 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:25.297579 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" podStartSLOduration=1.2975618629999999 podStartE2EDuration="1.297561863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:31:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:31:25.295785299 +0000 UTC m=+2205.542741566" watchObservedRunningTime="2026-07-23 10:31:25.297561863 +0000 UTC m=+2205.544518127" Jul 23 10:31:26.285863 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:26.285827 2580 generic.go:356] "Generic (PLEG): container finished" podID="bcea4dbf-546a-4737-8d38-9dfbb12f09a7" containerID="0b8ed734dafd240ae8fb638fcf9fe2f71dda803cb91992893467b8c5dbcad62c" exitCode=0 Jul 23 10:31:26.286287 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:26.285912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" event={"ID":"bcea4dbf-546a-4737-8d38-9dfbb12f09a7","Type":"ContainerDied","Data":"0b8ed734dafd240ae8fb638fcf9fe2f71dda803cb91992893467b8c5dbcad62c"} Jul 23 10:31:27.431240 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:27.431215 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" Jul 23 10:31:27.524215 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:27.524161 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcea4dbf-546a-4737-8d38-9dfbb12f09a7-maas-api-service-ca\") pod \"bcea4dbf-546a-4737-8d38-9dfbb12f09a7\" (UID: \"bcea4dbf-546a-4737-8d38-9dfbb12f09a7\") " Jul 23 10:31:27.524499 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:27.524477 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcea4dbf-546a-4737-8d38-9dfbb12f09a7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bcea4dbf-546a-4737-8d38-9dfbb12f09a7" (UID: "bcea4dbf-546a-4737-8d38-9dfbb12f09a7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 10:31:27.625143 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:27.625054 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcea4dbf-546a-4737-8d38-9dfbb12f09a7-maas-api-service-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:31:28.296401 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:28.296377 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" Jul 23 10:31:28.296401 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:28.296385 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476" event={"ID":"bcea4dbf-546a-4737-8d38-9dfbb12f09a7","Type":"ContainerDied","Data":"f77f172ae0a4a7a6621312996ba6774bcb5d4d8bef92831d77ad421fcaf8fceb"} Jul 23 10:31:28.296578 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:28.296426 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f77f172ae0a4a7a6621312996ba6774bcb5d4d8bef92831d77ad421fcaf8fceb" Jul 23 10:31:37.880606 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:37.880566 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:31:39.669300 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:39.669264 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476"] Jul 23 10:31:39.674413 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:39.674376 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6fb9c7-29476"] Jul 23 10:31:40.280625 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:40.280583 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcea4dbf-546a-4737-8d38-9dfbb12f09a7" path="/var/lib/kubelet/pods/bcea4dbf-546a-4737-8d38-9dfbb12f09a7/volumes" Jul 23 10:31:57.378999 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:31:57.378966 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:32:10.838575 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:10.838486 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg"] Jul 23 10:32:10.839031 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:10.838858 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcea4dbf-546a-4737-8d38-9dfbb12f09a7" containerName="revoke-keys" Jul 23 10:32:10.839031 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:10.838875 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcea4dbf-546a-4737-8d38-9dfbb12f09a7" containerName="revoke-keys" Jul 23 10:32:10.839031 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:10.839022 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcea4dbf-546a-4737-8d38-9dfbb12f09a7" containerName="revoke-keys" Jul 23 10:32:10.842453 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:10.842430 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg" Jul 23 10:32:10.844395 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:10.844366 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2vbpd\"" Jul 23 10:32:10.844512 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:10.844441 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 10:32:10.848936 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:10.848913 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg"] Jul 23 10:32:10.998816 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:10.998780 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76622c9b-ae8c-4d1f-bccc-99c7376b03d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg\" (UID: \"76622c9b-ae8c-4d1f-bccc-99c7376b03d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg" Jul 23 10:32:11.100311 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:11.100185 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76622c9b-ae8c-4d1f-bccc-99c7376b03d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg\" (UID: \"76622c9b-ae8c-4d1f-bccc-99c7376b03d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg" Jul 23 10:32:11.100812 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:11.100793 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76622c9b-ae8c-4d1f-bccc-99c7376b03d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg\" (UID: \"76622c9b-ae8c-4d1f-bccc-99c7376b03d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg" Jul 23 10:32:11.156466 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:11.156430 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg" Jul 23 10:32:11.296361 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:11.296332 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg"] Jul 23 10:32:11.298699 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:32:11.298673 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76622c9b_ae8c_4d1f_bccc_99c7376b03d5.slice/crio-54164ad6202c6fec09e7f3742e932ee2e0c797861bd7349e378fde5096a5bf14 WatchSource:0}: Error finding container 54164ad6202c6fec09e7f3742e932ee2e0c797861bd7349e378fde5096a5bf14: Status 404 returned error can't find the container with id 54164ad6202c6fec09e7f3742e932ee2e0c797861bd7349e378fde5096a5bf14 Jul 23 10:32:11.529246 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:11.529206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg" event={"ID":"76622c9b-ae8c-4d1f-bccc-99c7376b03d5","Type":"ContainerStarted","Data":"54164ad6202c6fec09e7f3742e932ee2e0c797861bd7349e378fde5096a5bf14"} Jul 23 10:32:12.535441 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:12.535400 2580 generic.go:356] "Generic (PLEG): container finished" podID="76622c9b-ae8c-4d1f-bccc-99c7376b03d5" containerID="5f34bc09f09ea293adab8d80ab4eee8d327425c7c9149c20873424560558ef1c" exitCode=0 Jul 23 10:32:12.535912 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:12.535494 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg" event={"ID":"76622c9b-ae8c-4d1f-bccc-99c7376b03d5","Type":"ContainerDied","Data":"5f34bc09f09ea293adab8d80ab4eee8d327425c7c9149c20873424560558ef1c"} Jul 23 10:32:13.091452 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:13.091411 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:32:13.673269 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:13.673233 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg" Jul 23 10:32:13.824124 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:13.824034 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76622c9b-ae8c-4d1f-bccc-99c7376b03d5-maas-api-service-ca\") pod \"76622c9b-ae8c-4d1f-bccc-99c7376b03d5\" (UID: \"76622c9b-ae8c-4d1f-bccc-99c7376b03d5\") " Jul 23 10:32:13.824459 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:13.824435 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76622c9b-ae8c-4d1f-bccc-99c7376b03d5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "76622c9b-ae8c-4d1f-bccc-99c7376b03d5" (UID: "76622c9b-ae8c-4d1f-bccc-99c7376b03d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 10:32:13.925615 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:13.925578 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76622c9b-ae8c-4d1f-bccc-99c7376b03d5-maas-api-service-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:32:14.547695 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:14.547613 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg" Jul 23 10:32:14.547695 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:14.547630 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg" event={"ID":"76622c9b-ae8c-4d1f-bccc-99c7376b03d5","Type":"ContainerDied","Data":"54164ad6202c6fec09e7f3742e932ee2e0c797861bd7349e378fde5096a5bf14"} Jul 23 10:32:14.547695 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:14.547679 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54164ad6202c6fec09e7f3742e932ee2e0c797861bd7349e378fde5096a5bf14" Jul 23 10:32:29.984374 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:29.984335 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:32:30.909579 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:30.909544 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg"] Jul 23 10:32:30.911709 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:30.911680 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b4fbcc-qnfdg"] Jul 23 10:32:32.279744 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:32.279700 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76622c9b-ae8c-4d1f-bccc-99c7376b03d5" path="/var/lib/kubelet/pods/76622c9b-ae8c-4d1f-bccc-99c7376b03d5/volumes" Jul 23 10:32:37.901922 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:37.901883 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc"] Jul 23 10:32:37.902334 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:37.902257 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76622c9b-ae8c-4d1f-bccc-99c7376b03d5" containerName="revoke-keys" Jul 23 10:32:37.902334 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:37.902270 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="76622c9b-ae8c-4d1f-bccc-99c7376b03d5" containerName="revoke-keys" Jul 23 10:32:37.902474 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:37.902358 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="76622c9b-ae8c-4d1f-bccc-99c7376b03d5" containerName="revoke-keys" Jul 23 10:32:37.905455 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:37.905437 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" Jul 23 10:32:37.907476 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:37.907458 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2vbpd\"" Jul 23 10:32:37.907569 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:37.907496 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 10:32:37.912817 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:37.912730 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc"] Jul 23 10:32:38.029261 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:38.029225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/331b221d-0e1c-4eec-9218-f3f663a43d7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2df747-q5cdc\" (UID: \"331b221d-0e1c-4eec-9218-f3f663a43d7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" Jul 23 10:32:38.130456 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:38.130419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/331b221d-0e1c-4eec-9218-f3f663a43d7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2df747-q5cdc\" (UID: \"331b221d-0e1c-4eec-9218-f3f663a43d7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" Jul 23 10:32:38.131117 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:38.131097 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/331b221d-0e1c-4eec-9218-f3f663a43d7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2df747-q5cdc\" (UID: \"331b221d-0e1c-4eec-9218-f3f663a43d7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" Jul 23 10:32:38.216758 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:38.216730 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" Jul 23 10:32:38.355434 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:38.355408 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc"] Jul 23 10:32:38.357520 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:32:38.357486 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod331b221d_0e1c_4eec_9218_f3f663a43d7d.slice/crio-d1e5d63eb50bc8c47843f783c8c6bd9742bf9799ba3132453cf18fe6dbd3cb86 WatchSource:0}: Error finding container d1e5d63eb50bc8c47843f783c8c6bd9742bf9799ba3132453cf18fe6dbd3cb86: Status 404 returned error can't find the container with id d1e5d63eb50bc8c47843f783c8c6bd9742bf9799ba3132453cf18fe6dbd3cb86 Jul 23 10:32:38.678112 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:38.678080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" event={"ID":"331b221d-0e1c-4eec-9218-f3f663a43d7d","Type":"ContainerStarted","Data":"30160134d470e418e94d79b047f8f2bbcaf169e846b91559f138ed94fce0a493"} Jul 23 10:32:38.678302 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:38.678129 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" event={"ID":"331b221d-0e1c-4eec-9218-f3f663a43d7d","Type":"ContainerStarted","Data":"d1e5d63eb50bc8c47843f783c8c6bd9742bf9799ba3132453cf18fe6dbd3cb86"} Jul 23 10:32:38.693978 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:38.693917 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" podStartSLOduration=1.693901023 podStartE2EDuration="1.693901023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:32:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:32:38.691417884 +0000 UTC m=+2278.938374142" watchObservedRunningTime="2026-07-23 10:32:38.693901023 +0000 UTC m=+2278.940857288" Jul 23 10:32:39.684447 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:39.684412 2580 generic.go:356] "Generic (PLEG): container finished" podID="331b221d-0e1c-4eec-9218-f3f663a43d7d" containerID="30160134d470e418e94d79b047f8f2bbcaf169e846b91559f138ed94fce0a493" exitCode=0 Jul 23 10:32:39.684892 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:39.684497 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" event={"ID":"331b221d-0e1c-4eec-9218-f3f663a43d7d","Type":"ContainerDied","Data":"30160134d470e418e94d79b047f8f2bbcaf169e846b91559f138ed94fce0a493"} Jul 23 10:32:40.820500 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:40.820475 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" Jul 23 10:32:40.956383 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:40.956355 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/331b221d-0e1c-4eec-9218-f3f663a43d7d-maas-api-service-ca\") pod \"331b221d-0e1c-4eec-9218-f3f663a43d7d\" (UID: \"331b221d-0e1c-4eec-9218-f3f663a43d7d\") " Jul 23 10:32:40.956747 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:40.956726 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/331b221d-0e1c-4eec-9218-f3f663a43d7d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "331b221d-0e1c-4eec-9218-f3f663a43d7d" (UID: "331b221d-0e1c-4eec-9218-f3f663a43d7d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 10:32:41.057841 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:41.057799 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/331b221d-0e1c-4eec-9218-f3f663a43d7d-maas-api-service-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:32:41.695782 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:41.695750 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" Jul 23 10:32:41.695782 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:41.695756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc" event={"ID":"331b221d-0e1c-4eec-9218-f3f663a43d7d","Type":"ContainerDied","Data":"d1e5d63eb50bc8c47843f783c8c6bd9742bf9799ba3132453cf18fe6dbd3cb86"} Jul 23 10:32:41.696001 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:41.695804 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1e5d63eb50bc8c47843f783c8c6bd9742bf9799ba3132453cf18fe6dbd3cb86" Jul 23 10:32:57.984219 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:57.984160 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc"] Jul 23 10:32:57.987434 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:57.987405 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2df747-q5cdc"] Jul 23 10:32:58.280702 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:58.280615 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="331b221d-0e1c-4eec-9218-f3f663a43d7d" path="/var/lib/kubelet/pods/331b221d-0e1c-4eec-9218-f3f663a43d7d/volumes" Jul 23 10:32:58.484960 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:32:58.484922 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:33:17.096602 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:17.096563 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:33:28.481321 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:28.481280 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:33:33.485700 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.485619 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk"] Jul 23 10:33:33.486089 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.485940 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="331b221d-0e1c-4eec-9218-f3f663a43d7d" containerName="revoke-keys" Jul 23 10:33:33.486089 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.485950 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="331b221d-0e1c-4eec-9218-f3f663a43d7d" containerName="revoke-keys" Jul 23 10:33:33.486089 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.486015 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="331b221d-0e1c-4eec-9218-f3f663a43d7d" containerName="revoke-keys" Jul 23 10:33:33.488900 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.488882 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.491348 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.491324 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d0ac3e\"/\"openshift-service-ca.crt\"" Jul 23 10:33:33.492020 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.491894 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d0ac3e\"/\"kube-root-ca.crt\"" Jul 23 10:33:33.492020 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.491982 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d0ac3e\"/\"default-dockercfg-pr2w9\"" Jul 23 10:33:33.492184 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.492031 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d0ac3e\"/\"auth-test-model-d0ac3e-kserve-self-signed-certs\"" Jul 23 10:33:33.498457 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.498431 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk"] Jul 23 10:33:33.580986 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.580948 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tls-certs\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.580986 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.580993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-model-cache\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.581269 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.581023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-dshm\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.581269 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.581066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59bqx\" (UniqueName: \"kubernetes.io/projected/7a84eb86-efdc-40ba-acc5-0bf023a85c99-kube-api-access-59bqx\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.581269 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.581123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-home\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.581269 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.581142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tmp-dir\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.681757 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.681720 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-model-cache\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.681757 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.681754 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-dshm\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.682013 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.681781 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59bqx\" (UniqueName: \"kubernetes.io/projected/7a84eb86-efdc-40ba-acc5-0bf023a85c99-kube-api-access-59bqx\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.682013 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.681820 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-home\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.682013 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.681835 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tmp-dir\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.682013 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.681875 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tls-certs\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.682265 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.682242 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-model-cache\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.682345 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.682303 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-home\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.682345 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.682315 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tmp-dir\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.684168 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.684143 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-dshm\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.684530 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.684511 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tls-certs\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.690712 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.690689 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59bqx\" (UniqueName: \"kubernetes.io/projected/7a84eb86-efdc-40ba-acc5-0bf023a85c99-kube-api-access-59bqx\") pod \"auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.800108 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.800024 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:33.934804 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.934778 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk"] Jul 23 10:33:33.936402 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:33:33.936375 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a84eb86_efdc_40ba_acc5_0bf023a85c99.slice/crio-eb956c883568e3b5c11f1cf3d74dec4435e0fcef07168919f9f6efc784955e11 WatchSource:0}: Error finding container eb956c883568e3b5c11f1cf3d74dec4435e0fcef07168919f9f6efc784955e11: Status 404 returned error can't find the container with id eb956c883568e3b5c11f1cf3d74dec4435e0fcef07168919f9f6efc784955e11 Jul 23 10:33:33.985643 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:33.985614 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" event={"ID":"7a84eb86-efdc-40ba-acc5-0bf023a85c99","Type":"ContainerStarted","Data":"eb956c883568e3b5c11f1cf3d74dec4435e0fcef07168919f9f6efc784955e11"} Jul 23 10:33:34.992063 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:34.992024 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" event={"ID":"7a84eb86-efdc-40ba-acc5-0bf023a85c99","Type":"ContainerStarted","Data":"9da9e324eb017a0cfe5dd527703e13adfebe0d106a7ecdc4da1b6eeec4a73d85"} Jul 23 10:33:35.010384 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:35.010339 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" podStartSLOduration=2.010323997 podStartE2EDuration="2.010323997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:33:35.007599072 +0000 UTC m=+2335.254555337" watchObservedRunningTime="2026-07-23 10:33:35.010323997 +0000 UTC m=+2335.257280241" Jul 23 10:33:36.586695 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:36.586662 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:33:43.800928 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:43.800876 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:43.800928 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:43.800942 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:43.813298 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:43.813273 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:33:44.050148 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:33:44.050111 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:34:32.676206 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:34:32.676160 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:34:40.401339 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:34:40.401312 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:34:40.410107 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:34:40.410079 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:34:43.683117 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:34:43.683080 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:35:02.479990 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:35:02.479906 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:35:16.591588 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:35:16.591551 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:35:33.681086 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:35:33.681046 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:35:52.377015 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:35:52.376974 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:36:06.183366 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:06.183335 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:36:17.712786 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:17.712754 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk"] Jul 23 10:36:22.981834 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:22.981798 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:36:32.733519 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:32.733408 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" podUID="7a84eb86-efdc-40ba-acc5-0bf023a85c99" containerName="main" containerID="cri-o://9da9e324eb017a0cfe5dd527703e13adfebe0d106a7ecdc4da1b6eeec4a73d85" gracePeriod=45 Jul 23 10:36:32.937835 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:32.937800 2580 generic.go:356] "Generic (PLEG): container finished" podID="7a84eb86-efdc-40ba-acc5-0bf023a85c99" containerID="9da9e324eb017a0cfe5dd527703e13adfebe0d106a7ecdc4da1b6eeec4a73d85" exitCode=0 Jul 23 10:36:32.938014 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:32.937881 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" event={"ID":"7a84eb86-efdc-40ba-acc5-0bf023a85c99","Type":"ContainerDied","Data":"9da9e324eb017a0cfe5dd527703e13adfebe0d106a7ecdc4da1b6eeec4a73d85"} Jul 23 10:36:32.989820 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:32.989746 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:36:33.066592 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.066558 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-home\") pod \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " Jul 23 10:36:33.066803 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.066625 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tls-certs\") pod \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " Jul 23 10:36:33.066803 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.066643 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-dshm\") pod \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " Jul 23 10:36:33.066803 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.066665 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tmp-dir\") pod \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " Jul 23 10:36:33.066803 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.066697 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59bqx\" (UniqueName: \"kubernetes.io/projected/7a84eb86-efdc-40ba-acc5-0bf023a85c99-kube-api-access-59bqx\") pod \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " Jul 23 10:36:33.066803 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.066723 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-model-cache\") pod \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\" (UID: \"7a84eb86-efdc-40ba-acc5-0bf023a85c99\") " Jul 23 10:36:33.067092 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.066883 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-home" (OuterVolumeSpecName: "home") pod "7a84eb86-efdc-40ba-acc5-0bf023a85c99" (UID: "7a84eb86-efdc-40ba-acc5-0bf023a85c99"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:36:33.067092 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.067004 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a84eb86-efdc-40ba-acc5-0bf023a85c99" (UID: "7a84eb86-efdc-40ba-acc5-0bf023a85c99"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:36:33.067176 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.067086 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a84eb86-efdc-40ba-acc5-0bf023a85c99" (UID: "7a84eb86-efdc-40ba-acc5-0bf023a85c99"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:36:33.069095 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.069064 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a84eb86-efdc-40ba-acc5-0bf023a85c99" (UID: "7a84eb86-efdc-40ba-acc5-0bf023a85c99"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 10:36:33.069095 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.069066 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-dshm" (OuterVolumeSpecName: "dshm") pod "7a84eb86-efdc-40ba-acc5-0bf023a85c99" (UID: "7a84eb86-efdc-40ba-acc5-0bf023a85c99"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:36:33.069269 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.069109 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a84eb86-efdc-40ba-acc5-0bf023a85c99-kube-api-access-59bqx" (OuterVolumeSpecName: "kube-api-access-59bqx") pod "7a84eb86-efdc-40ba-acc5-0bf023a85c99" (UID: "7a84eb86-efdc-40ba-acc5-0bf023a85c99"). InnerVolumeSpecName "kube-api-access-59bqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:36:33.167805 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.167766 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-home\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:36:33.167805 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.167798 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tls-certs\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:36:33.167805 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.167808 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-dshm\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:36:33.167805 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.167816 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-tmp-dir\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:36:33.168080 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.167825 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59bqx\" (UniqueName: \"kubernetes.io/projected/7a84eb86-efdc-40ba-acc5-0bf023a85c99-kube-api-access-59bqx\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:36:33.168080 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.167834 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a84eb86-efdc-40ba-acc5-0bf023a85c99-model-cache\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:36:33.944677 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.944650 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" Jul 23 10:36:33.945112 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.944650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk" event={"ID":"7a84eb86-efdc-40ba-acc5-0bf023a85c99","Type":"ContainerDied","Data":"eb956c883568e3b5c11f1cf3d74dec4435e0fcef07168919f9f6efc784955e11"} Jul 23 10:36:33.945112 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.944777 2580 scope.go:117] "RemoveContainer" containerID="9da9e324eb017a0cfe5dd527703e13adfebe0d106a7ecdc4da1b6eeec4a73d85" Jul 23 10:36:33.965072 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.965041 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk"] Jul 23 10:36:33.971071 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:33.971047 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/auth-test-model-d0ac3e-kserve-5dd4c6c7f5-8sfzk"] Jul 23 10:36:34.280201 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:34.280152 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a84eb86-efdc-40ba-acc5-0bf023a85c99" path="/var/lib/kubelet/pods/7a84eb86-efdc-40ba-acc5-0bf023a85c99/volumes" Jul 23 10:36:52.794326 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.794284 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc"] Jul 23 10:36:52.794821 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.794752 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a84eb86-efdc-40ba-acc5-0bf023a85c99" containerName="main" Jul 23 10:36:52.794821 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.794771 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a84eb86-efdc-40ba-acc5-0bf023a85c99" containerName="main" Jul 23 10:36:52.794943 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.794882 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a84eb86-efdc-40ba-acc5-0bf023a85c99" containerName="main" Jul 23 10:36:52.797547 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.797524 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.799907 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.799885 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d0ac3e\"/\"openshift-service-ca.crt\"" Jul 23 10:36:52.800466 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.800449 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d0ac3e\"/\"kube-root-ca.crt\"" Jul 23 10:36:52.800542 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.800464 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d0ac3e\"/\"default-dockercfg-pr2w9\"" Jul 23 10:36:52.800590 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.800464 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d0ac3e\"/\"sub-test-model-d0ac3e-kserve-self-signed-certs\"" Jul 23 10:36:52.810895 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.810869 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc"] Jul 23 10:36:52.828615 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.828584 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-model-cache\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.828750 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.828650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-home\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.828750 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.828682 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-tmp-dir\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.828750 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.828717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-dshm\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.828750 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.828735 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzljq\" (UniqueName: \"kubernetes.io/projected/b320dd86-5863-4a53-9d31-a165aeb29ca0-kube-api-access-vzljq\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.828884 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.828758 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b320dd86-5863-4a53-9d31-a165aeb29ca0-tls-certs\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.929166 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.929129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-tmp-dir\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.929388 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.929176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-dshm\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.929388 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.929327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzljq\" (UniqueName: \"kubernetes.io/projected/b320dd86-5863-4a53-9d31-a165aeb29ca0-kube-api-access-vzljq\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.929388 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.929375 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b320dd86-5863-4a53-9d31-a165aeb29ca0-tls-certs\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.929556 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.929415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-model-cache\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.929556 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.929498 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-home\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.929653 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.929618 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-tmp-dir\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.929778 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.929758 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-model-cache\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.929863 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.929844 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-home\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.931485 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.931465 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-dshm\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.931934 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.931916 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b320dd86-5863-4a53-9d31-a165aeb29ca0-tls-certs\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:52.937928 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:52.937905 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzljq\" (UniqueName: \"kubernetes.io/projected/b320dd86-5863-4a53-9d31-a165aeb29ca0-kube-api-access-vzljq\") pod \"sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:53.107099 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:53.107009 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:36:53.241669 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:53.241643 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc"] Jul 23 10:36:53.243892 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:36:53.243866 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb320dd86_5863_4a53_9d31_a165aeb29ca0.slice/crio-fc34623126cd62e67385c16065d8e66cc851549880a3352b20c47c4c3a805d5d WatchSource:0}: Error finding container fc34623126cd62e67385c16065d8e66cc851549880a3352b20c47c4c3a805d5d: Status 404 returned error can't find the container with id fc34623126cd62e67385c16065d8e66cc851549880a3352b20c47c4c3a805d5d Jul 23 10:36:54.052130 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:54.052093 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" event={"ID":"b320dd86-5863-4a53-9d31-a165aeb29ca0","Type":"ContainerStarted","Data":"877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429"} Jul 23 10:36:54.052130 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:54.052135 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" event={"ID":"b320dd86-5863-4a53-9d31-a165aeb29ca0","Type":"ContainerStarted","Data":"fc34623126cd62e67385c16065d8e66cc851549880a3352b20c47c4c3a805d5d"} Jul 23 10:36:54.070069 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:54.070015 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" podStartSLOduration=2.069999831 podStartE2EDuration="2.069999831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:36:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:36:54.067749517 +0000 UTC m=+2534.314705783" watchObservedRunningTime="2026-07-23 10:36:54.069999831 +0000 UTC m=+2534.316956097" Jul 23 10:36:56.087858 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:36:56.087815 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:37:03.107538 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:03.107505 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:37:03.107992 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:03.107599 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:37:03.120086 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:03.120060 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:37:04.116169 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:04.116137 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:37:11.781601 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:11.781564 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:37:26.084661 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:26.084627 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:37:40.504210 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:40.504154 2580 scope.go:117] "RemoveContainer" containerID="4814ae39f142ff3d460c3064ae7d4ff9ed9e3d5ac8152d623e53f6313e8fe224" Jul 23 10:37:40.513245 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:40.513223 2580 scope.go:117] "RemoveContainer" containerID="0b8ed734dafd240ae8fb638fcf9fe2f71dda803cb91992893467b8c5dbcad62c" Jul 23 10:37:42.955461 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:42.955427 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc"] Jul 23 10:37:49.589497 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:49.589457 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:37:57.977246 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:57.977167 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" podUID="b320dd86-5863-4a53-9d31-a165aeb29ca0" containerName="main" containerID="cri-o://877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429" gracePeriod=45 Jul 23 10:37:58.220321 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.220290 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:37:58.305505 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.305419 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-tmp-dir\") pod \"b320dd86-5863-4a53-9d31-a165aeb29ca0\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " Jul 23 10:37:58.305505 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.305477 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-dshm\") pod \"b320dd86-5863-4a53-9d31-a165aeb29ca0\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " Jul 23 10:37:58.305505 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.305495 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-home\") pod \"b320dd86-5863-4a53-9d31-a165aeb29ca0\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " Jul 23 10:37:58.305752 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.305543 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzljq\" (UniqueName: \"kubernetes.io/projected/b320dd86-5863-4a53-9d31-a165aeb29ca0-kube-api-access-vzljq\") pod \"b320dd86-5863-4a53-9d31-a165aeb29ca0\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " Jul 23 10:37:58.305752 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.305569 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b320dd86-5863-4a53-9d31-a165aeb29ca0-tls-certs\") pod \"b320dd86-5863-4a53-9d31-a165aeb29ca0\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " Jul 23 10:37:58.305752 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.305613 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-model-cache\") pod \"b320dd86-5863-4a53-9d31-a165aeb29ca0\" (UID: \"b320dd86-5863-4a53-9d31-a165aeb29ca0\") " Jul 23 10:37:58.305752 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.305677 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b320dd86-5863-4a53-9d31-a165aeb29ca0" (UID: "b320dd86-5863-4a53-9d31-a165aeb29ca0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:37:58.305913 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.305791 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-home" (OuterVolumeSpecName: "home") pod "b320dd86-5863-4a53-9d31-a165aeb29ca0" (UID: "b320dd86-5863-4a53-9d31-a165aeb29ca0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:37:58.305913 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.305829 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-tmp-dir\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:37:58.305913 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.305831 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-model-cache" (OuterVolumeSpecName: "model-cache") pod "b320dd86-5863-4a53-9d31-a165aeb29ca0" (UID: "b320dd86-5863-4a53-9d31-a165aeb29ca0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:37:58.307828 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.307801 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-dshm" (OuterVolumeSpecName: "dshm") pod "b320dd86-5863-4a53-9d31-a165aeb29ca0" (UID: "b320dd86-5863-4a53-9d31-a165aeb29ca0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:37:58.307942 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.307873 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b320dd86-5863-4a53-9d31-a165aeb29ca0-kube-api-access-vzljq" (OuterVolumeSpecName: "kube-api-access-vzljq") pod "b320dd86-5863-4a53-9d31-a165aeb29ca0" (UID: "b320dd86-5863-4a53-9d31-a165aeb29ca0"). InnerVolumeSpecName "kube-api-access-vzljq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:37:58.307992 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.307939 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b320dd86-5863-4a53-9d31-a165aeb29ca0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b320dd86-5863-4a53-9d31-a165aeb29ca0" (UID: "b320dd86-5863-4a53-9d31-a165aeb29ca0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 10:37:58.392408 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.392373 2580 generic.go:356] "Generic (PLEG): container finished" podID="b320dd86-5863-4a53-9d31-a165aeb29ca0" containerID="877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429" exitCode=0 Jul 23 10:37:58.392591 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.392416 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" Jul 23 10:37:58.392591 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.392463 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" event={"ID":"b320dd86-5863-4a53-9d31-a165aeb29ca0","Type":"ContainerDied","Data":"877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429"} Jul 23 10:37:58.392591 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.392507 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc" event={"ID":"b320dd86-5863-4a53-9d31-a165aeb29ca0","Type":"ContainerDied","Data":"fc34623126cd62e67385c16065d8e66cc851549880a3352b20c47c4c3a805d5d"} Jul 23 10:37:58.392591 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.392529 2580 scope.go:117] "RemoveContainer" containerID="877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429" Jul 23 10:37:58.402290 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.402272 2580 scope.go:117] "RemoveContainer" containerID="877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429" Jul 23 10:37:58.402546 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:37:58.402528 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429\": container with ID starting with 877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429 not found: ID does not exist" containerID="877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429" Jul 23 10:37:58.402607 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.402555 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429"} err="failed to get container status \"877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429\": rpc error: code = NotFound desc = could not find container \"877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429\": container with ID starting with 877cbbcad7fffdbbce4dcc97748332dababb53f1b582a008f0cf616f5be30429 not found: ID does not exist" Jul 23 10:37:58.406494 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.406474 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzljq\" (UniqueName: \"kubernetes.io/projected/b320dd86-5863-4a53-9d31-a165aeb29ca0-kube-api-access-vzljq\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:37:58.406580 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.406496 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b320dd86-5863-4a53-9d31-a165aeb29ca0-tls-certs\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:37:58.406580 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.406513 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-model-cache\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:37:58.406580 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.406523 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-dshm\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:37:58.406580 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.406534 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b320dd86-5863-4a53-9d31-a165aeb29ca0-home\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:37:58.414445 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.414423 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc"] Jul 23 10:37:58.418021 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:37:58.417997 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d0ac3e/sub-test-model-d0ac3e-kserve-76f6c54975-9tvmc"] Jul 23 10:38:00.084025 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:00.083991 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:38:00.280473 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:00.280439 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b320dd86-5863-4a53-9d31-a165aeb29ca0" path="/var/lib/kubelet/pods/b320dd86-5863-4a53-9d31-a165aeb29ca0/volumes" Jul 23 10:38:14.486063 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:14.485968 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:38:29.889661 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:29.889625 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:38:40.545114 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:40.545077 2580 scope.go:117] "RemoveContainer" containerID="5f34bc09f09ea293adab8d80ab4eee8d327425c7c9149c20873424560558ef1c" Jul 23 10:38:40.553488 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:40.553468 2580 scope.go:117] "RemoveContainer" containerID="30160134d470e418e94d79b047f8f2bbcaf169e846b91559f138ed94fce0a493" Jul 23 10:38:45.814354 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:45.814315 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:38:54.695228 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:54.695175 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7857-140642798877104"] Jul 23 10:38:54.695657 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:54.695526 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b320dd86-5863-4a53-9d31-a165aeb29ca0" containerName="main" Jul 23 10:38:54.695657 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:54.695537 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b320dd86-5863-4a53-9d31-a165aeb29ca0" containerName="main" Jul 23 10:38:54.695657 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:54.695590 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b320dd86-5863-4a53-9d31-a165aeb29ca0" containerName="main" Jul 23 10:38:54.698462 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:54.698445 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798877104" Jul 23 10:38:54.700438 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:54.700416 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dwjfr\"" Jul 23 10:38:54.705509 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:54.705483 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798877104"] Jul 23 10:38:54.808038 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:54.807999 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mbct\" (UniqueName: \"kubernetes.io/projected/ab08d178-a905-4452-8c32-bb45ab1103f4-kube-api-access-4mbct\") pod \"test-curl-7857-140642798877104\" (UID: \"ab08d178-a905-4452-8c32-bb45ab1103f4\") " pod="opendatahub/test-curl-7857-140642798877104" Jul 23 10:38:54.909218 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:54.909160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mbct\" (UniqueName: \"kubernetes.io/projected/ab08d178-a905-4452-8c32-bb45ab1103f4-kube-api-access-4mbct\") pod \"test-curl-7857-140642798877104\" (UID: \"ab08d178-a905-4452-8c32-bb45ab1103f4\") " pod="opendatahub/test-curl-7857-140642798877104" Jul 23 10:38:54.917395 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:54.917366 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mbct\" (UniqueName: \"kubernetes.io/projected/ab08d178-a905-4452-8c32-bb45ab1103f4-kube-api-access-4mbct\") pod \"test-curl-7857-140642798877104\" (UID: \"ab08d178-a905-4452-8c32-bb45ab1103f4\") " pod="opendatahub/test-curl-7857-140642798877104" Jul 23 10:38:55.009178 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:55.009129 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798877104" Jul 23 10:38:55.149171 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:55.149051 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798877104"] Jul 23 10:38:55.151886 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:38:55.151857 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab08d178_a905_4452_8c32_bb45ab1103f4.slice/crio-940b23201a9b90fa6f4f4fbf7877ca11db8b8c8c86472c30c8eb09e94f96e0ba WatchSource:0}: Error finding container 940b23201a9b90fa6f4f4fbf7877ca11db8b8c8c86472c30c8eb09e94f96e0ba: Status 404 returned error can't find the container with id 940b23201a9b90fa6f4f4fbf7877ca11db8b8c8c86472c30c8eb09e94f96e0ba Jul 23 10:38:55.153269 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:55.153250 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 10:38:55.701184 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:55.701141 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798877104" event={"ID":"ab08d178-a905-4452-8c32-bb45ab1103f4","Type":"ContainerStarted","Data":"940b23201a9b90fa6f4f4fbf7877ca11db8b8c8c86472c30c8eb09e94f96e0ba"} Jul 23 10:38:56.707906 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:56.707869 2580 generic.go:356] "Generic (PLEG): container finished" podID="ab08d178-a905-4452-8c32-bb45ab1103f4" containerID="8997aa806c6ae0adeeb07e14f8d23d41de313c54a36675913a3700468ecffcf9" exitCode=0 Jul 23 10:38:56.708408 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:56.707912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798877104" event={"ID":"ab08d178-a905-4452-8c32-bb45ab1103f4","Type":"ContainerDied","Data":"8997aa806c6ae0adeeb07e14f8d23d41de313c54a36675913a3700468ecffcf9"} Jul 23 10:38:57.852050 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:57.852023 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798877104" Jul 23 10:38:57.888062 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:57.888032 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7857-140642798877104_ab08d178-a905-4452-8c32-bb45ab1103f4/test-curl-7857-140642798877104/0.log" Jul 23 10:38:57.914243 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:57.914177 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7857-140642798877104"] Jul 23 10:38:57.919974 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:57.919942 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7857-140642798877104"] Jul 23 10:38:57.939780 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:57.939750 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mbct\" (UniqueName: \"kubernetes.io/projected/ab08d178-a905-4452-8c32-bb45ab1103f4-kube-api-access-4mbct\") pod \"ab08d178-a905-4452-8c32-bb45ab1103f4\" (UID: \"ab08d178-a905-4452-8c32-bb45ab1103f4\") " Jul 23 10:38:57.942114 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:57.942093 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab08d178-a905-4452-8c32-bb45ab1103f4-kube-api-access-4mbct" (OuterVolumeSpecName: "kube-api-access-4mbct") pod "ab08d178-a905-4452-8c32-bb45ab1103f4" (UID: "ab08d178-a905-4452-8c32-bb45ab1103f4"). InnerVolumeSpecName "kube-api-access-4mbct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:38:58.040637 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.040545 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mbct\" (UniqueName: \"kubernetes.io/projected/ab08d178-a905-4452-8c32-bb45ab1103f4-kube-api-access-4mbct\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:38:58.046936 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.046900 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7857-140642798877232"] Jul 23 10:38:58.047364 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.047345 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab08d178-a905-4452-8c32-bb45ab1103f4" containerName="test-curl-7857-140642798877104" Jul 23 10:38:58.047364 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.047365 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab08d178-a905-4452-8c32-bb45ab1103f4" containerName="test-curl-7857-140642798877104" Jul 23 10:38:58.047517 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.047466 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab08d178-a905-4452-8c32-bb45ab1103f4" containerName="test-curl-7857-140642798877104" Jul 23 10:38:58.050494 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.050471 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798877232" Jul 23 10:38:58.057411 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.057382 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798877232"] Jul 23 10:38:58.141736 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.141695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dvv5\" (UniqueName: \"kubernetes.io/projected/6a45601c-662e-4584-907e-05627de8f1d0-kube-api-access-7dvv5\") pod \"test-curl-7857-140642798877232\" (UID: \"6a45601c-662e-4584-907e-05627de8f1d0\") " pod="opendatahub/test-curl-7857-140642798877232" Jul 23 10:38:58.243161 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.243127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dvv5\" (UniqueName: \"kubernetes.io/projected/6a45601c-662e-4584-907e-05627de8f1d0-kube-api-access-7dvv5\") pod \"test-curl-7857-140642798877232\" (UID: \"6a45601c-662e-4584-907e-05627de8f1d0\") " pod="opendatahub/test-curl-7857-140642798877232" Jul 23 10:38:58.251652 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.251625 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dvv5\" (UniqueName: \"kubernetes.io/projected/6a45601c-662e-4584-907e-05627de8f1d0-kube-api-access-7dvv5\") pod \"test-curl-7857-140642798877232\" (UID: \"6a45601c-662e-4584-907e-05627de8f1d0\") " pod="opendatahub/test-curl-7857-140642798877232" Jul 23 10:38:58.280133 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.280097 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab08d178-a905-4452-8c32-bb45ab1103f4" path="/var/lib/kubelet/pods/ab08d178-a905-4452-8c32-bb45ab1103f4/volumes" Jul 23 10:38:58.360904 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.360797 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798877232" Jul 23 10:38:58.498079 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.498041 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798877232"] Jul 23 10:38:58.501201 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:38:58.501162 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a45601c_662e_4584_907e_05627de8f1d0.slice/crio-c17cd34b56af9e2e569d533f0ca380f171a534ece35e28515f530b098107b391 WatchSource:0}: Error finding container c17cd34b56af9e2e569d533f0ca380f171a534ece35e28515f530b098107b391: Status 404 returned error can't find the container with id c17cd34b56af9e2e569d533f0ca380f171a534ece35e28515f530b098107b391 Jul 23 10:38:58.725395 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.725368 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798877104" Jul 23 10:38:58.725608 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.725369 2580 scope.go:117] "RemoveContainer" containerID="8997aa806c6ae0adeeb07e14f8d23d41de313c54a36675913a3700468ecffcf9" Jul 23 10:38:58.726820 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:58.726625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798877232" event={"ID":"6a45601c-662e-4584-907e-05627de8f1d0","Type":"ContainerStarted","Data":"c17cd34b56af9e2e569d533f0ca380f171a534ece35e28515f530b098107b391"} Jul 23 10:38:59.734535 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:59.734501 2580 generic.go:356] "Generic (PLEG): container finished" podID="6a45601c-662e-4584-907e-05627de8f1d0" containerID="aa72eb6bddd2e62dc2fd2e60b0e960b223b5b9bb0966ec4811e94c8cc1bbefab" exitCode=0 Jul 23 10:38:59.735221 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:38:59.734591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798877232" event={"ID":"6a45601c-662e-4584-907e-05627de8f1d0","Type":"ContainerDied","Data":"aa72eb6bddd2e62dc2fd2e60b0e960b223b5b9bb0966ec4811e94c8cc1bbefab"} Jul 23 10:39:00.674786 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:00.674750 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:39:00.875331 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:00.875303 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798877232" Jul 23 10:39:00.895210 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:00.895165 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7857-140642798877232_6a45601c-662e-4584-907e-05627de8f1d0/test-curl-7857-140642798877232/0.log" Jul 23 10:39:00.918972 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:00.918935 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7857-140642798877232"] Jul 23 10:39:00.922451 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:00.922424 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7857-140642798877232"] Jul 23 10:39:00.965777 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:00.965748 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dvv5\" (UniqueName: \"kubernetes.io/projected/6a45601c-662e-4584-907e-05627de8f1d0-kube-api-access-7dvv5\") pod \"6a45601c-662e-4584-907e-05627de8f1d0\" (UID: \"6a45601c-662e-4584-907e-05627de8f1d0\") " Jul 23 10:39:00.968363 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:00.968332 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a45601c-662e-4584-907e-05627de8f1d0-kube-api-access-7dvv5" (OuterVolumeSpecName: "kube-api-access-7dvv5") pod "6a45601c-662e-4584-907e-05627de8f1d0" (UID: "6a45601c-662e-4584-907e-05627de8f1d0"). InnerVolumeSpecName "kube-api-access-7dvv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:39:01.066654 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.066618 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dvv5\" (UniqueName: \"kubernetes.io/projected/6a45601c-662e-4584-907e-05627de8f1d0-kube-api-access-7dvv5\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:39:01.094692 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.094657 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7857-140642798920736"] Jul 23 10:39:01.095037 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.095024 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a45601c-662e-4584-907e-05627de8f1d0" containerName="test-curl-7857-140642798877232" Jul 23 10:39:01.095085 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.095039 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a45601c-662e-4584-907e-05627de8f1d0" containerName="test-curl-7857-140642798877232" Jul 23 10:39:01.095131 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.095092 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a45601c-662e-4584-907e-05627de8f1d0" containerName="test-curl-7857-140642798877232" Jul 23 10:39:01.097647 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.097632 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:01.104117 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.104087 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798920736"] Jul 23 10:39:01.268593 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.268505 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82clb\" (UniqueName: \"kubernetes.io/projected/ac59d486-2a0a-41ec-9efc-13f2a045129d-kube-api-access-82clb\") pod \"test-curl-7857-140642798920736\" (UID: \"ac59d486-2a0a-41ec-9efc-13f2a045129d\") " pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:01.369500 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.369455 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82clb\" (UniqueName: \"kubernetes.io/projected/ac59d486-2a0a-41ec-9efc-13f2a045129d-kube-api-access-82clb\") pod \"test-curl-7857-140642798920736\" (UID: \"ac59d486-2a0a-41ec-9efc-13f2a045129d\") " pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:01.378121 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.378092 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82clb\" (UniqueName: \"kubernetes.io/projected/ac59d486-2a0a-41ec-9efc-13f2a045129d-kube-api-access-82clb\") pod \"test-curl-7857-140642798920736\" (UID: \"ac59d486-2a0a-41ec-9efc-13f2a045129d\") " pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:01.408911 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.408877 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:01.549554 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.549455 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798920736"] Jul 23 10:39:01.552425 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:39:01.552399 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac59d486_2a0a_41ec_9efc_13f2a045129d.slice/crio-10cd20ff7a42ba8907f90566f1423ca3612e2e2bd5e3f962816518c3925615e7 WatchSource:0}: Error finding container 10cd20ff7a42ba8907f90566f1423ca3612e2e2bd5e3f962816518c3925615e7: Status 404 returned error can't find the container with id 10cd20ff7a42ba8907f90566f1423ca3612e2e2bd5e3f962816518c3925615e7 Jul 23 10:39:01.745945 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.745911 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798920736" event={"ID":"ac59d486-2a0a-41ec-9efc-13f2a045129d","Type":"ContainerStarted","Data":"10cd20ff7a42ba8907f90566f1423ca3612e2e2bd5e3f962816518c3925615e7"} Jul 23 10:39:01.747160 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.747132 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798877232" Jul 23 10:39:01.747160 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:01.747147 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c17cd34b56af9e2e569d533f0ca380f171a534ece35e28515f530b098107b391" Jul 23 10:39:01.756808 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:39:01.756775 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7857-140642798877232\" is forbidden: User \"system:node:ip-10-0-139-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-55.ec2.internal' and this object" podUID="6a45601c-662e-4584-907e-05627de8f1d0" pod="opendatahub/test-curl-7857-140642798877232" Jul 23 10:39:02.279772 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:02.279739 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a45601c-662e-4584-907e-05627de8f1d0" path="/var/lib/kubelet/pods/6a45601c-662e-4584-907e-05627de8f1d0/volumes" Jul 23 10:39:02.753531 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:02.753494 2580 generic.go:356] "Generic (PLEG): container finished" podID="ac59d486-2a0a-41ec-9efc-13f2a045129d" containerID="eeccd089da5ea80cef73bb8578b679c56d68751201a48fd5ab0101e3d9b03033" exitCode=0 Jul 23 10:39:02.753694 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:02.753579 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798920736" event={"ID":"ac59d486-2a0a-41ec-9efc-13f2a045129d","Type":"ContainerDied","Data":"eeccd089da5ea80cef73bb8578b679c56d68751201a48fd5ab0101e3d9b03033"} Jul 23 10:39:03.890994 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:03.890970 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:03.909799 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:03.909773 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7857-140642798920736_ac59d486-2a0a-41ec-9efc-13f2a045129d/test-curl-7857-140642798920736/0.log" Jul 23 10:39:03.933566 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:03.933532 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7857-140642798920736"] Jul 23 10:39:03.939516 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:03.939488 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7857-140642798920736"] Jul 23 10:39:03.992463 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:03.992430 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82clb\" (UniqueName: \"kubernetes.io/projected/ac59d486-2a0a-41ec-9efc-13f2a045129d-kube-api-access-82clb\") pod \"ac59d486-2a0a-41ec-9efc-13f2a045129d\" (UID: \"ac59d486-2a0a-41ec-9efc-13f2a045129d\") " Jul 23 10:39:03.994698 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:03.994673 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac59d486-2a0a-41ec-9efc-13f2a045129d-kube-api-access-82clb" (OuterVolumeSpecName: "kube-api-access-82clb") pod "ac59d486-2a0a-41ec-9efc-13f2a045129d" (UID: "ac59d486-2a0a-41ec-9efc-13f2a045129d"). InnerVolumeSpecName "kube-api-access-82clb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:39:04.059379 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.059289 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7857-140642798920736"] Jul 23 10:39:04.059625 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.059612 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac59d486-2a0a-41ec-9efc-13f2a045129d" containerName="test-curl-7857-140642798920736" Jul 23 10:39:04.059668 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.059626 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac59d486-2a0a-41ec-9efc-13f2a045129d" containerName="test-curl-7857-140642798920736" Jul 23 10:39:04.059703 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.059688 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac59d486-2a0a-41ec-9efc-13f2a045129d" containerName="test-curl-7857-140642798920736" Jul 23 10:39:04.062600 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.062581 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:04.068638 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.068610 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798920736"] Jul 23 10:39:04.093416 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.093377 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82clb\" (UniqueName: \"kubernetes.io/projected/ac59d486-2a0a-41ec-9efc-13f2a045129d-kube-api-access-82clb\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:39:04.194182 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.194146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbm76\" (UniqueName: \"kubernetes.io/projected/322039d7-6961-411e-9f78-6fc150af02f5-kube-api-access-xbm76\") pod \"test-curl-7857-140642798920736\" (UID: \"322039d7-6961-411e-9f78-6fc150af02f5\") " pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:04.279964 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.279935 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac59d486-2a0a-41ec-9efc-13f2a045129d" path="/var/lib/kubelet/pods/ac59d486-2a0a-41ec-9efc-13f2a045129d/volumes" Jul 23 10:39:04.294804 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.294777 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbm76\" (UniqueName: \"kubernetes.io/projected/322039d7-6961-411e-9f78-6fc150af02f5-kube-api-access-xbm76\") pod \"test-curl-7857-140642798920736\" (UID: \"322039d7-6961-411e-9f78-6fc150af02f5\") " pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:04.302512 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.302491 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbm76\" (UniqueName: \"kubernetes.io/projected/322039d7-6961-411e-9f78-6fc150af02f5-kube-api-access-xbm76\") pod \"test-curl-7857-140642798920736\" (UID: \"322039d7-6961-411e-9f78-6fc150af02f5\") " pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:04.373250 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.373132 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:04.503397 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.503372 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798920736"] Jul 23 10:39:04.505385 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:39:04.505356 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod322039d7_6961_411e_9f78_6fc150af02f5.slice/crio-4b6dc8ddcf3871aa4efe212dada585d95ff51bf84823411585a7dd672681e0c2 WatchSource:0}: Error finding container 4b6dc8ddcf3871aa4efe212dada585d95ff51bf84823411585a7dd672681e0c2: Status 404 returned error can't find the container with id 4b6dc8ddcf3871aa4efe212dada585d95ff51bf84823411585a7dd672681e0c2 Jul 23 10:39:04.764121 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.764091 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798920736" event={"ID":"322039d7-6961-411e-9f78-6fc150af02f5","Type":"ContainerStarted","Data":"4b6dc8ddcf3871aa4efe212dada585d95ff51bf84823411585a7dd672681e0c2"} Jul 23 10:39:04.765366 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.765341 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:04.765462 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:04.765383 2580 scope.go:117] "RemoveContainer" containerID="eeccd089da5ea80cef73bb8578b679c56d68751201a48fd5ab0101e3d9b03033" Jul 23 10:39:05.772913 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:05.772876 2580 generic.go:356] "Generic (PLEG): container finished" podID="322039d7-6961-411e-9f78-6fc150af02f5" containerID="36e57bbc93c580d795e809f7ff2eecad8d454b80842e5f2c89eeb0cab8b8b88a" exitCode=0 Jul 23 10:39:05.773394 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:05.772961 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798920736" event={"ID":"322039d7-6961-411e-9f78-6fc150af02f5","Type":"ContainerDied","Data":"36e57bbc93c580d795e809f7ff2eecad8d454b80842e5f2c89eeb0cab8b8b88a"} Jul 23 10:39:06.914760 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:06.914737 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:06.936990 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:06.936959 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7857-140642798920736_322039d7-6961-411e-9f78-6fc150af02f5/test-curl-7857-140642798920736/0.log" Jul 23 10:39:06.960119 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:06.960078 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7857-140642798920736"] Jul 23 10:39:06.961999 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:06.961975 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7857-140642798920736"] Jul 23 10:39:07.019082 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.019045 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbm76\" (UniqueName: \"kubernetes.io/projected/322039d7-6961-411e-9f78-6fc150af02f5-kube-api-access-xbm76\") pod \"322039d7-6961-411e-9f78-6fc150af02f5\" (UID: \"322039d7-6961-411e-9f78-6fc150af02f5\") " Jul 23 10:39:07.021393 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.021362 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/322039d7-6961-411e-9f78-6fc150af02f5-kube-api-access-xbm76" (OuterVolumeSpecName: "kube-api-access-xbm76") pod "322039d7-6961-411e-9f78-6fc150af02f5" (UID: "322039d7-6961-411e-9f78-6fc150af02f5"). InnerVolumeSpecName "kube-api-access-xbm76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:39:07.085358 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.085276 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7857-140642798921744"] Jul 23 10:39:07.085617 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.085605 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="322039d7-6961-411e-9f78-6fc150af02f5" containerName="test-curl-7857-140642798920736" Jul 23 10:39:07.085658 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.085618 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="322039d7-6961-411e-9f78-6fc150af02f5" containerName="test-curl-7857-140642798920736" Jul 23 10:39:07.085693 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.085688 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="322039d7-6961-411e-9f78-6fc150af02f5" containerName="test-curl-7857-140642798920736" Jul 23 10:39:07.088688 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.088669 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:07.094606 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.094578 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798921744"] Jul 23 10:39:07.120399 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.120370 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbm76\" (UniqueName: \"kubernetes.io/projected/322039d7-6961-411e-9f78-6fc150af02f5-kube-api-access-xbm76\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:39:07.221265 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.221227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w95bw\" (UniqueName: \"kubernetes.io/projected/b9600b21-01db-4424-af29-1481a7ae180d-kube-api-access-w95bw\") pod \"test-curl-7857-140642798921744\" (UID: \"b9600b21-01db-4424-af29-1481a7ae180d\") " pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:07.322548 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.322507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w95bw\" (UniqueName: \"kubernetes.io/projected/b9600b21-01db-4424-af29-1481a7ae180d-kube-api-access-w95bw\") pod \"test-curl-7857-140642798921744\" (UID: \"b9600b21-01db-4424-af29-1481a7ae180d\") " pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:07.332158 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.332126 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w95bw\" (UniqueName: \"kubernetes.io/projected/b9600b21-01db-4424-af29-1481a7ae180d-kube-api-access-w95bw\") pod \"test-curl-7857-140642798921744\" (UID: \"b9600b21-01db-4424-af29-1481a7ae180d\") " pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:07.399295 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.399182 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:07.534258 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.534178 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798921744"] Jul 23 10:39:07.536341 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:39:07.536290 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9600b21_01db_4424_af29_1481a7ae180d.slice/crio-7896c837edfba81f81461d99300595686d0bb85f86509da2f3f090ae568e8374 WatchSource:0}: Error finding container 7896c837edfba81f81461d99300595686d0bb85f86509da2f3f090ae568e8374: Status 404 returned error can't find the container with id 7896c837edfba81f81461d99300595686d0bb85f86509da2f3f090ae568e8374 Jul 23 10:39:07.785677 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.785640 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798921744" event={"ID":"b9600b21-01db-4424-af29-1481a7ae180d","Type":"ContainerStarted","Data":"7896c837edfba81f81461d99300595686d0bb85f86509da2f3f090ae568e8374"} Jul 23 10:39:07.786787 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.786771 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:07.786899 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:07.786816 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b6dc8ddcf3871aa4efe212dada585d95ff51bf84823411585a7dd672681e0c2" Jul 23 10:39:07.796513 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:39:07.796484 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7857-140642798920736\" is forbidden: User \"system:node:ip-10-0-139-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-55.ec2.internal' and this object" podUID="322039d7-6961-411e-9f78-6fc150af02f5" pod="opendatahub/test-curl-7857-140642798920736" Jul 23 10:39:08.280750 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:08.280714 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="322039d7-6961-411e-9f78-6fc150af02f5" path="/var/lib/kubelet/pods/322039d7-6961-411e-9f78-6fc150af02f5/volumes" Jul 23 10:39:08.794984 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:08.794940 2580 generic.go:356] "Generic (PLEG): container finished" podID="b9600b21-01db-4424-af29-1481a7ae180d" containerID="e55480b78671fa7e73d8cb6d74b245e0da63892c6c84c47010070ae8c32128f3" exitCode=0 Jul 23 10:39:08.794984 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:08.794979 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798921744" event={"ID":"b9600b21-01db-4424-af29-1481a7ae180d","Type":"ContainerDied","Data":"e55480b78671fa7e73d8cb6d74b245e0da63892c6c84c47010070ae8c32128f3"} Jul 23 10:39:09.940434 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:09.940406 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:09.961441 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:09.961414 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7857-140642798921744_b9600b21-01db-4424-af29-1481a7ae180d/test-curl-7857-140642798921744/0.log" Jul 23 10:39:09.984038 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:09.983997 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7857-140642798921744"] Jul 23 10:39:09.987344 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:09.987314 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7857-140642798921744"] Jul 23 10:39:10.048555 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.048519 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w95bw\" (UniqueName: \"kubernetes.io/projected/b9600b21-01db-4424-af29-1481a7ae180d-kube-api-access-w95bw\") pod \"b9600b21-01db-4424-af29-1481a7ae180d\" (UID: \"b9600b21-01db-4424-af29-1481a7ae180d\") " Jul 23 10:39:10.050727 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.050702 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9600b21-01db-4424-af29-1481a7ae180d-kube-api-access-w95bw" (OuterVolumeSpecName: "kube-api-access-w95bw") pod "b9600b21-01db-4424-af29-1481a7ae180d" (UID: "b9600b21-01db-4424-af29-1481a7ae180d"). InnerVolumeSpecName "kube-api-access-w95bw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:39:10.105713 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.105678 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7857-140642798921744"] Jul 23 10:39:10.106016 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.106002 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9600b21-01db-4424-af29-1481a7ae180d" containerName="test-curl-7857-140642798921744" Jul 23 10:39:10.106054 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.106017 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9600b21-01db-4424-af29-1481a7ae180d" containerName="test-curl-7857-140642798921744" Jul 23 10:39:10.106089 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.106081 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9600b21-01db-4424-af29-1481a7ae180d" containerName="test-curl-7857-140642798921744" Jul 23 10:39:10.108997 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.108980 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:10.113133 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.113101 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798921744"] Jul 23 10:39:10.149815 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.149781 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w95bw\" (UniqueName: \"kubernetes.io/projected/b9600b21-01db-4424-af29-1481a7ae180d-kube-api-access-w95bw\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:39:10.250898 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.250864 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jgcw\" (UniqueName: \"kubernetes.io/projected/f44f681a-495b-4e67-ac47-727ab826ab7a-kube-api-access-9jgcw\") pod \"test-curl-7857-140642798921744\" (UID: \"f44f681a-495b-4e67-ac47-727ab826ab7a\") " pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:10.281797 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.281760 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9600b21-01db-4424-af29-1481a7ae180d" path="/var/lib/kubelet/pods/b9600b21-01db-4424-af29-1481a7ae180d/volumes" Jul 23 10:39:10.351913 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.351808 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jgcw\" (UniqueName: \"kubernetes.io/projected/f44f681a-495b-4e67-ac47-727ab826ab7a-kube-api-access-9jgcw\") pod \"test-curl-7857-140642798921744\" (UID: \"f44f681a-495b-4e67-ac47-727ab826ab7a\") " pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:10.359799 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.359767 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jgcw\" (UniqueName: \"kubernetes.io/projected/f44f681a-495b-4e67-ac47-727ab826ab7a-kube-api-access-9jgcw\") pod \"test-curl-7857-140642798921744\" (UID: \"f44f681a-495b-4e67-ac47-727ab826ab7a\") " pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:10.418953 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.418919 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:10.555019 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.554989 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-140642798921744"] Jul 23 10:39:10.556817 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:39:10.556792 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf44f681a_495b_4e67_ac47_727ab826ab7a.slice/crio-08ae09bf97b033063bfe59dd242eaceed03c745f0147048ad10825284f716f74 WatchSource:0}: Error finding container 08ae09bf97b033063bfe59dd242eaceed03c745f0147048ad10825284f716f74: Status 404 returned error can't find the container with id 08ae09bf97b033063bfe59dd242eaceed03c745f0147048ad10825284f716f74 Jul 23 10:39:10.808084 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.808059 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:10.808307 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.808089 2580 scope.go:117] "RemoveContainer" containerID="e55480b78671fa7e73d8cb6d74b245e0da63892c6c84c47010070ae8c32128f3" Jul 23 10:39:10.809504 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:10.809475 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798921744" event={"ID":"f44f681a-495b-4e67-ac47-727ab826ab7a","Type":"ContainerStarted","Data":"08ae09bf97b033063bfe59dd242eaceed03c745f0147048ad10825284f716f74"} Jul 23 10:39:11.816782 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:11.816741 2580 generic.go:356] "Generic (PLEG): container finished" podID="f44f681a-495b-4e67-ac47-727ab826ab7a" containerID="458f7520f787b7e3972446133da952d64a22e52498e6ead98b01fb58168891d4" exitCode=0 Jul 23 10:39:11.817209 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:11.816798 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-140642798921744" event={"ID":"f44f681a-495b-4e67-ac47-727ab826ab7a","Type":"ContainerDied","Data":"458f7520f787b7e3972446133da952d64a22e52498e6ead98b01fb58168891d4"} Jul 23 10:39:12.958639 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:12.958612 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:12.974911 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:12.974880 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jgcw\" (UniqueName: \"kubernetes.io/projected/f44f681a-495b-4e67-ac47-727ab826ab7a-kube-api-access-9jgcw\") pod \"f44f681a-495b-4e67-ac47-727ab826ab7a\" (UID: \"f44f681a-495b-4e67-ac47-727ab826ab7a\") " Jul 23 10:39:12.977471 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:12.977426 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f44f681a-495b-4e67-ac47-727ab826ab7a-kube-api-access-9jgcw" (OuterVolumeSpecName: "kube-api-access-9jgcw") pod "f44f681a-495b-4e67-ac47-727ab826ab7a" (UID: "f44f681a-495b-4e67-ac47-727ab826ab7a"). InnerVolumeSpecName "kube-api-access-9jgcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:39:12.981652 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:12.981626 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7857-140642798921744_f44f681a-495b-4e67-ac47-727ab826ab7a/test-curl-7857-140642798921744/0.log" Jul 23 10:39:13.005596 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.005563 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7857-140642798921744"] Jul 23 10:39:13.009021 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.008992 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7857-140642798921744"] Jul 23 10:39:13.076435 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.076350 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jgcw\" (UniqueName: \"kubernetes.io/projected/f44f681a-495b-4e67-ac47-727ab826ab7a-kube-api-access-9jgcw\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:39:13.187011 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.186976 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4"] Jul 23 10:39:13.187325 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.187313 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f44f681a-495b-4e67-ac47-727ab826ab7a" containerName="test-curl-7857-140642798921744" Jul 23 10:39:13.187368 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.187327 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f44f681a-495b-4e67-ac47-727ab826ab7a" containerName="test-curl-7857-140642798921744" Jul 23 10:39:13.187414 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.187380 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f44f681a-495b-4e67-ac47-727ab826ab7a" containerName="test-curl-7857-140642798921744" Jul 23 10:39:13.190091 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.190075 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4" Jul 23 10:39:13.191831 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.191805 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2vbpd\"" Jul 23 10:39:13.191960 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.191854 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 10:39:13.197775 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.197519 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4"] Jul 23 10:39:13.278440 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.278400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70b27a0d-2642-4e0f-aab1-475e22b07c50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4\" (UID: \"70b27a0d-2642-4e0f-aab1-475e22b07c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4" Jul 23 10:39:13.379116 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.379016 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70b27a0d-2642-4e0f-aab1-475e22b07c50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4\" (UID: \"70b27a0d-2642-4e0f-aab1-475e22b07c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4" Jul 23 10:39:13.379605 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.379587 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70b27a0d-2642-4e0f-aab1-475e22b07c50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4\" (UID: \"70b27a0d-2642-4e0f-aab1-475e22b07c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4" Jul 23 10:39:13.501074 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.501039 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4" Jul 23 10:39:13.640546 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.640514 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4"] Jul 23 10:39:13.642681 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:39:13.642656 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70b27a0d_2642_4e0f_aab1_475e22b07c50.slice/crio-48af7590836a680ca2605e9e15e10565c189ded32e4ad58baf84001e82e07f07 WatchSource:0}: Error finding container 48af7590836a680ca2605e9e15e10565c189ded32e4ad58baf84001e82e07f07: Status 404 returned error can't find the container with id 48af7590836a680ca2605e9e15e10565c189ded32e4ad58baf84001e82e07f07 Jul 23 10:39:13.829564 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.829541 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:13.829729 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.829608 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08ae09bf97b033063bfe59dd242eaceed03c745f0147048ad10825284f716f74" Jul 23 10:39:13.831076 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:13.831039 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4" event={"ID":"70b27a0d-2642-4e0f-aab1-475e22b07c50","Type":"ContainerStarted","Data":"48af7590836a680ca2605e9e15e10565c189ded32e4ad58baf84001e82e07f07"} Jul 23 10:39:13.847094 ip-10-0-139-55 kubenswrapper[2580]: E0723 10:39:13.847066 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7857-140642798921744\" is forbidden: User \"system:node:ip-10-0-139-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-55.ec2.internal' and this object" podUID="f44f681a-495b-4e67-ac47-727ab826ab7a" pod="opendatahub/test-curl-7857-140642798921744" Jul 23 10:39:14.279723 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:14.279688 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f44f681a-495b-4e67-ac47-727ab826ab7a" path="/var/lib/kubelet/pods/f44f681a-495b-4e67-ac47-727ab826ab7a/volumes" Jul 23 10:39:14.837545 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:14.837507 2580 generic.go:356] "Generic (PLEG): container finished" podID="70b27a0d-2642-4e0f-aab1-475e22b07c50" containerID="7d5cfc7f93b546ab4e537988100dd4889b8419e2c7ac5551faae749e0b69b851" exitCode=0 Jul 23 10:39:14.837725 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:14.837596 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4" event={"ID":"70b27a0d-2642-4e0f-aab1-475e22b07c50","Type":"ContainerDied","Data":"7d5cfc7f93b546ab4e537988100dd4889b8419e2c7ac5551faae749e0b69b851"} Jul 23 10:39:15.978162 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:15.978134 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4" Jul 23 10:39:15.999103 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:15.999066 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70b27a0d-2642-4e0f-aab1-475e22b07c50-maas-api-service-ca\") pod \"70b27a0d-2642-4e0f-aab1-475e22b07c50\" (UID: \"70b27a0d-2642-4e0f-aab1-475e22b07c50\") " Jul 23 10:39:15.999475 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:15.999450 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70b27a0d-2642-4e0f-aab1-475e22b07c50-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "70b27a0d-2642-4e0f-aab1-475e22b07c50" (UID: "70b27a0d-2642-4e0f-aab1-475e22b07c50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 10:39:16.100465 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:16.100380 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70b27a0d-2642-4e0f-aab1-475e22b07c50-maas-api-service-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:39:16.849361 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:16.849330 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4" Jul 23 10:39:16.849361 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:16.849345 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4" event={"ID":"70b27a0d-2642-4e0f-aab1-475e22b07c50","Type":"ContainerDied","Data":"48af7590836a680ca2605e9e15e10565c189ded32e4ad58baf84001e82e07f07"} Jul 23 10:39:16.849591 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:16.849395 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48af7590836a680ca2605e9e15e10565c189ded32e4ad58baf84001e82e07f07" Jul 23 10:39:26.882151 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:26.882112 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:39:28.263844 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:28.263793 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4"] Jul 23 10:39:28.265773 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:28.265752 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d0ac3e-k5cg4"] Jul 23 10:39:28.281045 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:28.281011 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70b27a0d-2642-4e0f-aab1-475e22b07c50" path="/var/lib/kubelet/pods/70b27a0d-2642-4e0f-aab1-475e22b07c50/volumes" Jul 23 10:39:35.236846 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.236760 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf"] Jul 23 10:39:35.237268 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.237140 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70b27a0d-2642-4e0f-aab1-475e22b07c50" containerName="revoke-keys" Jul 23 10:39:35.237268 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.237152 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="70b27a0d-2642-4e0f-aab1-475e22b07c50" containerName="revoke-keys" Jul 23 10:39:35.237268 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.237219 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="70b27a0d-2642-4e0f-aab1-475e22b07c50" containerName="revoke-keys" Jul 23 10:39:35.241926 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.241907 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf" Jul 23 10:39:35.244383 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.244357 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 10:39:35.244711 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.244357 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2vbpd\"" Jul 23 10:39:35.246606 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.246579 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf"] Jul 23 10:39:35.349863 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.349818 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e1ab3a-e405-4da1-bab2-c32da020445d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf\" (UID: \"78e1ab3a-e405-4da1-bab2-c32da020445d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf" Jul 23 10:39:35.450737 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.450695 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e1ab3a-e405-4da1-bab2-c32da020445d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf\" (UID: \"78e1ab3a-e405-4da1-bab2-c32da020445d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf" Jul 23 10:39:35.451511 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.451477 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e1ab3a-e405-4da1-bab2-c32da020445d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf\" (UID: \"78e1ab3a-e405-4da1-bab2-c32da020445d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf" Jul 23 10:39:35.553238 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.553129 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf" Jul 23 10:39:35.686955 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.686930 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf"] Jul 23 10:39:35.689686 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:39:35.689656 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78e1ab3a_e405_4da1_bab2_c32da020445d.slice/crio-2e2ee286e31608b223be8e6eac447469fd5fc05ad7f3083e718202475532e56d WatchSource:0}: Error finding container 2e2ee286e31608b223be8e6eac447469fd5fc05ad7f3083e718202475532e56d: Status 404 returned error can't find the container with id 2e2ee286e31608b223be8e6eac447469fd5fc05ad7f3083e718202475532e56d Jul 23 10:39:35.947456 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:35.947366 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf" event={"ID":"78e1ab3a-e405-4da1-bab2-c32da020445d","Type":"ContainerStarted","Data":"2e2ee286e31608b223be8e6eac447469fd5fc05ad7f3083e718202475532e56d"} Jul 23 10:39:36.954454 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:36.954416 2580 generic.go:356] "Generic (PLEG): container finished" podID="78e1ab3a-e405-4da1-bab2-c32da020445d" containerID="6205b2d9b2cfc3e16ea0f50ecce0fb2fb8474d41a776b8f5eceeb71ef748084b" exitCode=0 Jul 23 10:39:36.954850 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:36.954513 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf" event={"ID":"78e1ab3a-e405-4da1-bab2-c32da020445d","Type":"ContainerDied","Data":"6205b2d9b2cfc3e16ea0f50ecce0fb2fb8474d41a776b8f5eceeb71ef748084b"} Jul 23 10:39:38.091286 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:38.091261 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf" Jul 23 10:39:38.175482 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:38.175448 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e1ab3a-e405-4da1-bab2-c32da020445d-maas-api-service-ca\") pod \"78e1ab3a-e405-4da1-bab2-c32da020445d\" (UID: \"78e1ab3a-e405-4da1-bab2-c32da020445d\") " Jul 23 10:39:38.175802 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:38.175779 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78e1ab3a-e405-4da1-bab2-c32da020445d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "78e1ab3a-e405-4da1-bab2-c32da020445d" (UID: "78e1ab3a-e405-4da1-bab2-c32da020445d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 10:39:38.276209 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:38.276109 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e1ab3a-e405-4da1-bab2-c32da020445d-maas-api-service-ca\") on node \"ip-10-0-139-55.ec2.internal\" DevicePath \"\"" Jul 23 10:39:38.966466 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:38.966437 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf" Jul 23 10:39:38.966644 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:38.966479 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf" event={"ID":"78e1ab3a-e405-4da1-bab2-c32da020445d","Type":"ContainerDied","Data":"2e2ee286e31608b223be8e6eac447469fd5fc05ad7f3083e718202475532e56d"} Jul 23 10:39:38.966644 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:38.966527 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e2ee286e31608b223be8e6eac447469fd5fc05ad7f3083e718202475532e56d" Jul 23 10:39:40.433568 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:40.433538 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:39:40.444821 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:40.444792 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q4p8r_12d1431e-93fe-4b29-9dbc-eae156e6f45e/ovn-acl-logging/0.log" Jul 23 10:39:48.579932 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:48.579897 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b5268"] Jul 23 10:39:50.300782 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:50.300742 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf"] Jul 23 10:39:50.307714 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:50.304756 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bc0d57-mcltf"] Jul 23 10:39:52.280023 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:52.279986 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78e1ab3a-e405-4da1-bab2-c32da020445d" path="/var/lib/kubelet/pods/78e1ab3a-e405-4da1-bab2-c32da020445d/volumes" Jul 23 10:39:57.466039 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:39:57.466004 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6466556dd9-jxx7j_739ec57c-e508-4d7b-b18a-8f3b7ea49b92/authorino/0.log" Jul 23 10:40:03.579845 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:03.579810 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bc8b458c-2zwn9_43617c9f-db83-4f72-ac31-4320dc43aa7f/copy-manifests/0.log" Jul 23 10:40:03.598075 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:03.598042 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bc8b458c-2zwn9_43617c9f-db83-4f72-ac31-4320dc43aa7f/manager/0.log" Jul 23 10:40:04.441650 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:04.441619 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29746710-lhmmp_7fd7f04c-5055-43bb-b204-301bbe9fdbb6/cleanup/0.log" Jul 23 10:40:05.382349 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.382319 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl_0f567380-cc4e-4790-acac-7bc69e32816b/util/0.log" Jul 23 10:40:05.388688 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.388665 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl_0f567380-cc4e-4790-acac-7bc69e32816b/pull/0.log" Jul 23 10:40:05.393862 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.393843 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl_0f567380-cc4e-4790-acac-7bc69e32816b/extract/0.log" Jul 23 10:40:05.514985 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.514955 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd_9bb321af-33a1-4857-9e9b-060fa06aaa3f/util/0.log" Jul 23 10:40:05.521356 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.521321 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd_9bb321af-33a1-4857-9e9b-060fa06aaa3f/pull/0.log" Jul 23 10:40:05.527474 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.527421 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd_9bb321af-33a1-4857-9e9b-060fa06aaa3f/extract/0.log" Jul 23 10:40:05.647940 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.647850 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z_6d95e9ed-7a0f-435f-b671-ad119c756bfa/util/0.log" Jul 23 10:40:05.655064 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.655040 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z_6d95e9ed-7a0f-435f-b671-ad119c756bfa/pull/0.log" Jul 23 10:40:05.661925 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.661893 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z_6d95e9ed-7a0f-435f-b671-ad119c756bfa/extract/0.log" Jul 23 10:40:05.777395 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.777368 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd_f9fcb11b-d40f-4a55-b464-b8566329b086/util/0.log" Jul 23 10:40:05.786119 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.786094 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd_f9fcb11b-d40f-4a55-b464-b8566329b086/pull/0.log" Jul 23 10:40:05.792251 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.792214 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd_f9fcb11b-d40f-4a55-b464-b8566329b086/extract/0.log" Jul 23 10:40:05.933621 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:05.933540 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6466556dd9-jxx7j_739ec57c-e508-4d7b-b18a-8f3b7ea49b92/authorino/0.log" Jul 23 10:40:06.290734 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:06.290708 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-g4vcs_10496e28-71fa-4e14-b42e-d87c0b903bf4/kuadrant-console-plugin/0.log" Jul 23 10:40:06.659065 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:06.658982 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b5268_e30101a3-6590-47c8-977f-ba5d8503f742/limitador/0.log" Jul 23 10:40:07.532788 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:07.532759 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565557bd5f-xpngg_21248380-e684-4a60-b641-fc605527e60c/kube-auth-proxy/0.log" Jul 23 10:40:08.623914 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:08.623883 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-bnvgk_08d2ea1d-1238-417f-bc32-a59a50645324/main/0.log" Jul 23 10:40:14.121036 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.120998 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zjjkv/must-gather-wvchr"] Jul 23 10:40:14.121528 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.121368 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e1ab3a-e405-4da1-bab2-c32da020445d" containerName="revoke-keys" Jul 23 10:40:14.121528 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.121379 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e1ab3a-e405-4da1-bab2-c32da020445d" containerName="revoke-keys" Jul 23 10:40:14.121528 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.121446 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="78e1ab3a-e405-4da1-bab2-c32da020445d" containerName="revoke-keys" Jul 23 10:40:14.125723 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.125701 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zjjkv/must-gather-wvchr" Jul 23 10:40:14.127964 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.127943 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zjjkv\"/\"default-dockercfg-7rpwf\"" Jul 23 10:40:14.128053 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.127943 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zjjkv\"/\"openshift-service-ca.crt\"" Jul 23 10:40:14.128417 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.128404 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zjjkv\"/\"kube-root-ca.crt\"" Jul 23 10:40:14.139328 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.139304 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zjjkv/must-gather-wvchr"] Jul 23 10:40:14.288792 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.288758 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9xb7\" (UniqueName: \"kubernetes.io/projected/7d47b711-77ba-41e6-8049-8f43d323032c-kube-api-access-m9xb7\") pod \"must-gather-wvchr\" (UID: \"7d47b711-77ba-41e6-8049-8f43d323032c\") " pod="openshift-must-gather-zjjkv/must-gather-wvchr" Jul 23 10:40:14.288987 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.288835 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7d47b711-77ba-41e6-8049-8f43d323032c-must-gather-output\") pod \"must-gather-wvchr\" (UID: \"7d47b711-77ba-41e6-8049-8f43d323032c\") " pod="openshift-must-gather-zjjkv/must-gather-wvchr" Jul 23 10:40:14.389439 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.389355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9xb7\" (UniqueName: \"kubernetes.io/projected/7d47b711-77ba-41e6-8049-8f43d323032c-kube-api-access-m9xb7\") pod \"must-gather-wvchr\" (UID: \"7d47b711-77ba-41e6-8049-8f43d323032c\") " pod="openshift-must-gather-zjjkv/must-gather-wvchr" Jul 23 10:40:14.389595 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.389452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7d47b711-77ba-41e6-8049-8f43d323032c-must-gather-output\") pod \"must-gather-wvchr\" (UID: \"7d47b711-77ba-41e6-8049-8f43d323032c\") " pod="openshift-must-gather-zjjkv/must-gather-wvchr" Jul 23 10:40:14.389738 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.389721 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7d47b711-77ba-41e6-8049-8f43d323032c-must-gather-output\") pod \"must-gather-wvchr\" (UID: \"7d47b711-77ba-41e6-8049-8f43d323032c\") " pod="openshift-must-gather-zjjkv/must-gather-wvchr" Jul 23 10:40:14.399836 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.399811 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9xb7\" (UniqueName: \"kubernetes.io/projected/7d47b711-77ba-41e6-8049-8f43d323032c-kube-api-access-m9xb7\") pod \"must-gather-wvchr\" (UID: \"7d47b711-77ba-41e6-8049-8f43d323032c\") " pod="openshift-must-gather-zjjkv/must-gather-wvchr" Jul 23 10:40:14.435916 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.435881 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zjjkv/must-gather-wvchr" Jul 23 10:40:14.582969 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:14.582948 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zjjkv/must-gather-wvchr"] Jul 23 10:40:14.585418 ip-10-0-139-55 kubenswrapper[2580]: W0723 10:40:14.585389 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d47b711_77ba_41e6_8049_8f43d323032c.slice/crio-d95b9350921f4f14810338c48ef90165e8703487d3a67a91840c93532f90c97e WatchSource:0}: Error finding container d95b9350921f4f14810338c48ef90165e8703487d3a67a91840c93532f90c97e: Status 404 returned error can't find the container with id d95b9350921f4f14810338c48ef90165e8703487d3a67a91840c93532f90c97e Jul 23 10:40:15.163610 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:15.163580 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zjjkv/must-gather-wvchr" event={"ID":"7d47b711-77ba-41e6-8049-8f43d323032c","Type":"ContainerStarted","Data":"d95b9350921f4f14810338c48ef90165e8703487d3a67a91840c93532f90c97e"} Jul 23 10:40:16.172498 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:16.171930 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zjjkv/must-gather-wvchr" event={"ID":"7d47b711-77ba-41e6-8049-8f43d323032c","Type":"ContainerStarted","Data":"304b52ebb52b20556aa793dc38b9308fb62f98224020917109cbfde985b45d84"} Jul 23 10:40:16.172498 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:16.171990 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zjjkv/must-gather-wvchr" event={"ID":"7d47b711-77ba-41e6-8049-8f43d323032c","Type":"ContainerStarted","Data":"571dd700daeecf34ebeaf2bbd93efaf3e3d7385b3a2aaf50b71eeda6e2aeef34"} Jul 23 10:40:16.189733 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:16.189661 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zjjkv/must-gather-wvchr" podStartSLOduration=1.2621031280000001 podStartE2EDuration="2.18963747s" totalImagesPullingTime="927.534342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:40:16.187488987 +0000 UTC m=+2736.434445254" watchObservedRunningTime="2026-07-23 10:40:16.18963747 +0000 UTC m=+2736.436593738" Jul 23 10:40:17.176719 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:17.176686 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vw75n_566f324b-9a5b-4829-8ed1-69b00afb4c0d/global-pull-secret-syncer/0.log" Jul 23 10:40:17.327070 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:17.327033 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dkm2f_5baa7946-10bb-42ca-bd72-bd64914f2d91/konnectivity-agent/0.log" Jul 23 10:40:17.397153 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:17.397116 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-55.ec2.internal_d71d6e1134f838d6a77fe78231efcb8f/haproxy/0.log" Jul 23 10:40:21.888926 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:21.888883 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl_0f567380-cc4e-4790-acac-7bc69e32816b/extract/0.log" Jul 23 10:40:21.920183 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:21.920149 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl_0f567380-cc4e-4790-acac-7bc69e32816b/util/0.log" Jul 23 10:40:21.947975 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:21.947925 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75987bdl_0f567380-cc4e-4790-acac-7bc69e32816b/pull/0.log" Jul 23 10:40:21.979788 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:21.979739 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd_9bb321af-33a1-4857-9e9b-060fa06aaa3f/extract/0.log" Jul 23 10:40:22.006740 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:22.006707 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd_9bb321af-33a1-4857-9e9b-060fa06aaa3f/util/0.log" Jul 23 10:40:22.032760 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:22.032726 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vj9xd_9bb321af-33a1-4857-9e9b-060fa06aaa3f/pull/0.log" Jul 23 10:40:22.083601 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:22.083564 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z_6d95e9ed-7a0f-435f-b671-ad119c756bfa/extract/0.log" Jul 23 10:40:22.136607 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:22.136579 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z_6d95e9ed-7a0f-435f-b671-ad119c756bfa/util/0.log" Jul 23 10:40:22.161239 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:22.161109 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735672z_6d95e9ed-7a0f-435f-b671-ad119c756bfa/pull/0.log" Jul 23 10:40:22.192574 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:22.192533 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd_f9fcb11b-d40f-4a55-b464-b8566329b086/extract/0.log" Jul 23 10:40:22.222663 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:22.222629 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd_f9fcb11b-d40f-4a55-b464-b8566329b086/util/0.log" Jul 23 10:40:22.254005 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:22.253974 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bp6gd_f9fcb11b-d40f-4a55-b464-b8566329b086/pull/0.log" Jul 23 10:40:22.898216 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:22.897275 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6466556dd9-jxx7j_739ec57c-e508-4d7b-b18a-8f3b7ea49b92/authorino/0.log" Jul 23 10:40:23.008413 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:23.008380 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-g4vcs_10496e28-71fa-4e14-b42e-d87c0b903bf4/kuadrant-console-plugin/0.log" Jul 23 10:40:23.289815 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:23.289784 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b5268_e30101a3-6590-47c8-977f-ba5d8503f742/limitador/0.log" Jul 23 10:40:25.355008 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:25.354965 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-blfnp_6cd20837-ec25-45d3-b289-69747da7303d/kube-state-metrics/0.log" Jul 23 10:40:25.379363 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:25.379324 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-blfnp_6cd20837-ec25-45d3-b289-69747da7303d/kube-rbac-proxy-main/0.log" Jul 23 10:40:25.405919 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:25.405886 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-blfnp_6cd20837-ec25-45d3-b289-69747da7303d/kube-rbac-proxy-self/0.log" Jul 23 10:40:25.544158 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:25.544130 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dwjpc_c10d452e-f9a5-4eb2-8493-0eefe0a0e848/node-exporter/0.log" Jul 23 10:40:25.568970 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:25.568945 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dwjpc_c10d452e-f9a5-4eb2-8493-0eefe0a0e848/kube-rbac-proxy/0.log" Jul 23 10:40:25.596907 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:25.596882 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dwjpc_c10d452e-f9a5-4eb2-8493-0eefe0a0e848/init-textfile/0.log" Jul 23 10:40:26.202725 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:26.202690 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-zt8mz_2f86cbde-44fe-42d8-a26e-0df389d7fb3e/prometheus-operator/0.log" Jul 23 10:40:26.223563 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:26.223522 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-zt8mz_2f86cbde-44fe-42d8-a26e-0df389d7fb3e/kube-rbac-proxy/0.log" Jul 23 10:40:26.254726 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:26.254699 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-8n7z8_9697c23f-7286-4092-86c3-41b8fb851d67/prometheus-operator-admission-webhook/0.log" Jul 23 10:40:26.436635 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:26.436598 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c766dcc55-vxm4v_3173cb8c-7f47-48df-a9fd-c3f3004237cd/thanos-query/0.log" Jul 23 10:40:26.460380 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:26.460286 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c766dcc55-vxm4v_3173cb8c-7f47-48df-a9fd-c3f3004237cd/kube-rbac-proxy-web/0.log" Jul 23 10:40:26.482560 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:26.482533 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c766dcc55-vxm4v_3173cb8c-7f47-48df-a9fd-c3f3004237cd/kube-rbac-proxy/0.log" Jul 23 10:40:26.503480 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:26.503447 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c766dcc55-vxm4v_3173cb8c-7f47-48df-a9fd-c3f3004237cd/prom-label-proxy/0.log" Jul 23 10:40:26.523560 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:26.523510 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c766dcc55-vxm4v_3173cb8c-7f47-48df-a9fd-c3f3004237cd/kube-rbac-proxy-rules/0.log" Jul 23 10:40:26.548211 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:26.548168 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c766dcc55-vxm4v_3173cb8c-7f47-48df-a9fd-c3f3004237cd/kube-rbac-proxy-metrics/0.log" Jul 23 10:40:27.869481 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:27.869434 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg"] Jul 23 10:40:27.875851 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:27.875812 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:27.885365 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:27.884974 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg"] Jul 23 10:40:27.937608 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:27.937565 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-proc\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:27.937947 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:27.937926 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7qth\" (UniqueName: \"kubernetes.io/projected/c1250429-0e62-4b91-9975-45669b9b48a8-kube-api-access-d7qth\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:27.938115 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:27.938102 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-lib-modules\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:27.938269 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:27.938255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-podres\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:27.938441 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:27.938426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-sys\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.039750 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.039715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-proc\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.039946 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.039775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7qth\" (UniqueName: \"kubernetes.io/projected/c1250429-0e62-4b91-9975-45669b9b48a8-kube-api-access-d7qth\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.039946 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.039810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-lib-modules\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.039946 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.039827 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-proc\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.039946 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.039843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-podres\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.039946 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.039917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-sys\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.040239 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.039986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-sys\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.040239 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.040000 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-lib-modules\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.040239 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.040156 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c1250429-0e62-4b91-9975-45669b9b48a8-podres\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.048077 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.048053 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7qth\" (UniqueName: \"kubernetes.io/projected/c1250429-0e62-4b91-9975-45669b9b48a8-kube-api-access-d7qth\") pod \"perf-node-gather-daemonset-h2xvg\" (UID: \"c1250429-0e62-4b91-9975-45669b9b48a8\") " pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.191402 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.191241 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:28.379133 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.377331 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg"] Jul 23 10:40:28.958685 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.958655 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7c57fb7b78-gchrv_1951a0ec-3156-49cf-8fb1-9ec47d86fbd5/console/0.log" Jul 23 10:40:28.999236 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:28.999178 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-9zqpw_b94460bd-0bb8-4b55-aed9-d59dc00cd0ae/download-server/0.log" Jul 23 10:40:29.282910 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:29.282820 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" event={"ID":"c1250429-0e62-4b91-9975-45669b9b48a8","Type":"ContainerStarted","Data":"aaa7ee83c3eb58fc376f2977b6f9627466d77a3d96865e7886ec1ca347917301"} Jul 23 10:40:29.282910 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:29.282882 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:29.282910 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:29.282900 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" event={"ID":"c1250429-0e62-4b91-9975-45669b9b48a8","Type":"ContainerStarted","Data":"4091f13923f1b983e82e6894ab07f1e3a8886df76d591f824a956ee1247948a2"} Jul 23 10:40:29.299615 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:29.299547 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" podStartSLOduration=2.299523283 podStartE2EDuration="2.299523283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 10:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:40:29.298866264 +0000 UTC m=+2749.545822599" watchObservedRunningTime="2026-07-23 10:40:29.299523283 +0000 UTC m=+2749.546479549" Jul 23 10:40:30.655465 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:30.655437 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4csjr_3511d554-433a-49ed-87c4-fcf369aa0bf8/dns/0.log" Jul 23 10:40:30.674274 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:30.674244 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4csjr_3511d554-433a-49ed-87c4-fcf369aa0bf8/kube-rbac-proxy/0.log" Jul 23 10:40:30.815861 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:30.815828 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lcx56_4a666fcc-084b-4948-8fac-8b7824ed1b62/dns-node-resolver/0.log" Jul 23 10:40:31.429015 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:31.428960 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kbzkh_c83bbb36-0921-4e7e-a036-c6fad9d60e43/node-ca/0.log" Jul 23 10:40:32.664218 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:32.664171 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565557bd5f-xpngg_21248380-e684-4a60-b641-fc605527e60c/kube-auth-proxy/0.log" Jul 23 10:40:33.544371 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:33.544336 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sc4hx_33558acb-61b9-4b54-9c54-4ef58db51b48/serve-healthcheck-canary/0.log" Jul 23 10:40:34.164780 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:34.164744 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rn7mv_4cb72c19-cc93-4b00-b759-464689c41963/kube-rbac-proxy/0.log" Jul 23 10:40:34.184878 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:34.184854 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rn7mv_4cb72c19-cc93-4b00-b759-464689c41963/exporter/0.log" Jul 23 10:40:34.208681 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:34.208650 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rn7mv_4cb72c19-cc93-4b00-b759-464689c41963/extractor/0.log" Jul 23 10:40:35.303059 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:35.303029 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zjjkv/perf-node-gather-daemonset-h2xvg" Jul 23 10:40:37.320668 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:37.320637 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bc8b458c-2zwn9_43617c9f-db83-4f72-ac31-4320dc43aa7f/manager/0.log" Jul 23 10:40:37.343588 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:37.343555 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bc8b458c-2zwn9_43617c9f-db83-4f72-ac31-4320dc43aa7f/copy-manifests/0.log" Jul 23 10:40:38.607025 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:38.606992 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565557bd5f-xpngg_21248380-e684-4a60-b641-fc605527e60c/kube-auth-proxy/0.log" Jul 23 10:40:38.791335 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:38.791307 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d6fb6dcb8-8jmw5_2a5d3298-81cb-40f8-8dab-62d57711652c/manager/0.log" Jul 23 10:40:38.849125 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:38.849067 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-sj7kh_5a53d146-b071-4217-ad9d-a7bc9f1cf7e3/openshift-lws-operator/0.log" Jul 23 10:40:40.328654 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:40.328612 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565557bd5f-xpngg_21248380-e684-4a60-b641-fc605527e60c/kube-auth-proxy/0.log" Jul 23 10:40:45.256179 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:45.256103 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5dvjs_f1a41df2-3624-4d86-9851-cb67975b09ac/kube-multus/0.log" Jul 23 10:40:45.429393 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:45.429346 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9k5p_ef627b06-7dad-475b-b4b0-2908731467e1/kube-multus-additional-cni-plugins/0.log" Jul 23 10:40:45.452531 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:45.452503 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9k5p_ef627b06-7dad-475b-b4b0-2908731467e1/egress-router-binary-copy/0.log" Jul 23 10:40:45.473147 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:45.473109 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9k5p_ef627b06-7dad-475b-b4b0-2908731467e1/cni-plugins/0.log" Jul 23 10:40:45.492703 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:45.492674 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9k5p_ef627b06-7dad-475b-b4b0-2908731467e1/bond-cni-plugin/0.log" Jul 23 10:40:45.511908 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:45.511831 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9k5p_ef627b06-7dad-475b-b4b0-2908731467e1/routeoverride-cni/0.log" Jul 23 10:40:45.531317 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:45.531287 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9k5p_ef627b06-7dad-475b-b4b0-2908731467e1/whereabouts-cni-bincopy/0.log" Jul 23 10:40:45.552604 ip-10-0-139-55 kubenswrapper[2580]: I0723 10:40:45.552575 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h9k5p_ef627b06-7dad-475b-b4b0-2908731467e1/whereabouts-cni/0.log"