Jul 06 23:55:06.039304 ip-10-0-130-52 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 23:55:06.039319 ip-10-0-130-52 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 23:55:06.039329 ip-10-0-130-52 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 23:55:06.039690 ip-10-0-130-52 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 23:55:16.228069 ip-10-0-130-52 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 23:55:16.228088 ip-10-0-130-52 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b6976b4844f44540b3b8356f13783863 -- Jul 06 23:57:12.144705 ip-10-0-130-52 systemd[1]: Starting Kubernetes Kubelet... Jul 06 23:57:14.339532 ip-10-0-130-52 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:57:14.339532 ip-10-0-130-52 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 23:57:14.339532 ip-10-0-130-52 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:57:14.339532 ip-10-0-130-52 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 23:57:14.339532 ip-10-0-130-52 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:57:14.341329 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.341232 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 23:57:14.346891 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346866 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:57:14.346891 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346889 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:57:14.346891 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346892 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346897 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346900 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346903 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346906 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346908 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346911 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346914 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346917 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346919 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346922 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346925 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346928 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346930 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346933 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346936 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346939 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346942 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346951 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346953 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:57:14.346993 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346956 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346959 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346962 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346964 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346967 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346970 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346972 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346975 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346978 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346981 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346983 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346988 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346992 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.346995 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347003 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347006 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347009 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347013 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347017 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347020 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:57:14.347502 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347023 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347027 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347030 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347033 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347036 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347039 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347042 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347045 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347047 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347050 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347053 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347055 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347058 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347060 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347075 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347079 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347081 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347084 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347087 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347089 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:57:14.348012 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347092 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347094 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347097 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347100 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347102 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347105 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347108 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347110 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347112 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347115 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347118 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347120 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347123 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347126 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347131 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347133 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347136 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347138 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347141 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:57:14.348499 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347143 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347146 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347148 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347151 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347153 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347156 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347159 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347163 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347166 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347169 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347172 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347175 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347178 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347181 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347184 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347186 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347189 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:57:14.348955 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.347192 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:57:14.350239 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350222 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:57:14.350239 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350237 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:57:14.350239 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350241 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350244 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350247 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350250 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350253 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350256 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350258 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350262 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350264 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350268 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350270 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350273 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350276 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350279 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350281 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350286 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350289 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350293 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350295 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:57:14.350330 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350299 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350302 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350305 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350308 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350311 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350314 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350317 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350320 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350322 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350324 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350327 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350330 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350333 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350335 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350338 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350341 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350344 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350346 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350349 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350351 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350354 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:57:14.350828 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350358 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350362 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350366 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350369 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350372 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350374 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350377 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350379 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350382 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350385 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350387 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350390 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350392 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350396 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350399 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350401 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350404 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350407 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350409 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:57:14.351348 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350412 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350414 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350417 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350420 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350422 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350424 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350427 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350429 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350432 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350434 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350437 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350439 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350442 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350444 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350447 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350451 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350453 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350456 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350459 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350462 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:57:14.351817 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350464 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350467 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350470 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350472 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350475 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350477 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350480 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350483 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350486 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350488 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350491 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350493 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350496 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350499 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350501 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350504 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350506 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350509 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350951 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:57:14.352305 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350957 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350960 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350963 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350966 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350969 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350972 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350975 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350978 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350981 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350983 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350986 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350989 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350991 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350994 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.350997 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351000 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351002 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351005 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351008 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351010 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:57:14.352764 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351013 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351015 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351018 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351021 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351023 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351026 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351029 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351031 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351034 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351036 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351039 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351042 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351044 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351047 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351050 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351052 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351055 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351057 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351060 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351079 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:57:14.353259 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351085 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351088 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351091 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351094 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351097 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351100 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351103 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351105 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351108 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351110 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351113 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351116 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351120 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351124 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351127 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351130 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351132 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351135 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351138 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351140 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:57:14.353739 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351143 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351146 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351148 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351152 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351157 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351160 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351163 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351166 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351169 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351172 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351174 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351180 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351183 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351187 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351190 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351193 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351196 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351198 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351201 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:57:14.354243 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351203 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351206 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351208 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351211 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351214 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351216 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351219 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351222 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351224 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351226 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351229 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351232 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351234 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351237 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351239 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351242 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351244 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351246 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351249 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351326 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 23:57:14.354695 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351335 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351347 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351352 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351357 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351361 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351366 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351371 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351375 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351378 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351381 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351385 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351388 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351391 2562 flags.go:64] FLAG: --cgroup-root="" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351394 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351397 2562 flags.go:64] FLAG: --client-ca-file="" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351400 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351403 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351408 2562 flags.go:64] FLAG: --cluster-domain="" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351410 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351413 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351417 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351420 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351423 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 23:57:14.355201 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351427 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351430 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351433 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351436 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351439 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351441 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351444 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351447 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351451 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351455 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351458 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351460 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351463 2562 flags.go:64] FLAG: --enable-server="true" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351467 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351472 2562 flags.go:64] FLAG: --event-burst="100" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351475 2562 flags.go:64] FLAG: --event-qps="50" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351478 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351482 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351485 2562 flags.go:64] FLAG: --eviction-hard="" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351489 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351492 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351494 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351497 2562 flags.go:64] FLAG: --eviction-soft="" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351500 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351503 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 23:57:14.355737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351506 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351509 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351512 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351515 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351518 2562 flags.go:64] FLAG: --feature-gates="" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351522 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351525 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351528 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351531 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351534 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351537 2562 flags.go:64] FLAG: --help="false" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351539 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351542 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351545 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351548 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351551 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351557 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351560 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351562 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351566 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351568 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351571 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351574 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351577 2562 flags.go:64] FLAG: --kube-reserved="" Jul 06 23:57:14.356424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351580 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351583 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351586 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351589 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351592 2562 flags.go:64] FLAG: --lock-file="" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351595 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351597 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351600 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351606 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351614 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351617 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351620 2562 flags.go:64] FLAG: --logging-format="text" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351623 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351626 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351629 2562 flags.go:64] FLAG: --manifest-url="" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351632 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351636 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351639 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351644 2562 flags.go:64] FLAG: --max-pods="110" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351647 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351649 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351653 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351656 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351659 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351663 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 23:57:14.357170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351666 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351674 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351677 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351680 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351683 2562 flags.go:64] FLAG: --pod-cidr="" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351686 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351692 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351695 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351698 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351702 2562 flags.go:64] FLAG: --port="10250" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351706 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351709 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02829681868080305" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351712 2562 flags.go:64] FLAG: --qos-reserved="" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351715 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351718 2562 flags.go:64] FLAG: --register-node="true" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351722 2562 flags.go:64] FLAG: --register-with-taints="" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351725 2562 flags.go:64] FLAG: --registry-burst="10" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351728 2562 flags.go:64] FLAG: --registry-qps="5" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351731 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351734 2562 flags.go:64] FLAG: --reserved-memory="" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351738 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351741 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351744 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351747 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351750 2562 flags.go:64] FLAG: --runonce="false" Jul 06 23:57:14.357781 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351752 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351755 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351759 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351762 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351764 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351767 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351779 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351786 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351789 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351792 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351795 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351798 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351801 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351804 2562 flags.go:64] FLAG: --system-cgroups="" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351807 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351813 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351815 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351819 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351825 2562 flags.go:64] FLAG: --tls-min-version="" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351828 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351831 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351834 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351836 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351839 2562 flags.go:64] FLAG: --v="2" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351843 2562 flags.go:64] FLAG: --version="false" Jul 06 23:57:14.358390 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351847 2562 flags.go:64] FLAG: --vmodule="" Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351851 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.351855 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351985 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351989 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351992 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351995 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.351999 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352003 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352006 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352008 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352011 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352015 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352021 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352025 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352028 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352031 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352034 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352037 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:57:14.358990 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352040 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352043 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352046 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352049 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352051 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352054 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352057 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352061 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352075 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352078 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352081 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352083 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352086 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352089 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352091 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352094 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352098 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352101 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352105 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:57:14.359469 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352108 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352111 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352114 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352116 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352119 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352122 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352125 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352129 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352133 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352136 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352139 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352142 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352144 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352147 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352150 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352152 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352155 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352158 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352161 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352163 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:57:14.359982 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352166 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352169 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352172 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352174 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352177 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352179 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352182 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352185 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352187 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352190 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352192 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352195 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352197 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352200 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352203 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352205 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352208 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352210 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352213 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352217 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352221 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:57:14.360482 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352224 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352226 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352229 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352231 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352234 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352237 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352239 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352242 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352245 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352247 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352250 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352253 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352256 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352258 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352261 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352264 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352266 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352269 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352272 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352274 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:57:14.360988 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352277 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352280 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.352282 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.353103 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.360040 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.360060 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.360100 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.360106 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360168 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360174 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360178 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360181 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360185 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360188 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360191 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:57:14.361573 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360194 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360197 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360199 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360202 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360205 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360208 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360211 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360213 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360216 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360218 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360221 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360224 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360227 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360230 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360232 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360235 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360237 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360240 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360242 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:57:14.361941 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360245 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360247 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360250 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360254 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360257 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360260 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360263 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360265 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360268 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360271 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360273 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360277 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360279 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360282 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360284 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360287 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360289 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360292 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360295 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360297 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:57:14.362402 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360300 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360303 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360305 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360308 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360311 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360314 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360316 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360319 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360321 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360324 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360326 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360329 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360333 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360337 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360340 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360343 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360347 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360350 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360353 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:57:14.362895 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360355 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360358 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360361 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360364 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360366 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360369 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360371 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360375 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360379 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360381 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360384 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360387 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360390 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360392 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360395 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360398 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360401 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360403 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360406 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360409 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:57:14.363373 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360411 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360413 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360416 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360418 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360421 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360423 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360426 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360429 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360433 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360437 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360440 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360442 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360444 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360447 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.360452 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:57:14.363865 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360573 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360579 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360582 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360585 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360588 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360591 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360594 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360597 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360600 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360604 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360606 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360609 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360613 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360615 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360618 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360621 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360623 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360626 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360628 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360631 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:57:14.364245 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360633 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360636 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360638 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360641 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360645 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360647 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360650 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360652 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360658 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360663 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360667 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360670 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360673 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360676 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360679 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360681 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360684 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360687 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360689 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:57:14.364743 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360693 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360696 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360699 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360701 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360704 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360706 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360709 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360713 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360716 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360719 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360721 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360724 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360727 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360729 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360732 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360734 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360737 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360740 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360742 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:57:14.365285 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360745 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360747 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360750 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360753 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360756 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360758 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360761 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360764 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360766 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360768 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360771 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360773 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360776 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360778 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360781 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360783 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360786 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360788 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360791 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360793 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360797 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:57:14.365744 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360799 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360802 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360804 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360807 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360809 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360812 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360814 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360817 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360819 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360822 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360824 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360826 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360829 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360832 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360834 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360837 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360839 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360842 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360844 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:57:14.366258 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:14.360847 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:57:14.366715 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.360852 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:57:14.366715 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.360992 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 23:57:14.366715 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.363093 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 23:57:14.366715 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.364495 2562 server.go:1019] "Starting client certificate rotation" Jul 06 23:57:14.366715 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.364605 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:57:14.366715 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.364634 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:57:14.367290 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.367274 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:57:14.368828 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.368810 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:57:14.383237 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.383211 2562 log.go:25] "Validated CRI v1 runtime API" Jul 06 23:57:14.384982 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.384964 2562 log.go:25] "Validated CRI v1 image API" Jul 06 23:57:14.385858 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.385837 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 23:57:14.389694 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.389670 2562 fs.go:135] Filesystem UUIDs: map[3bbd1ea1-0796-4288-8585-46f2b7b4f5b8:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 980501c6-4915-4c1e-8727-48b908c28b85:/dev/nvme0n1p4] Jul 06 23:57:14.389790 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.389691 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 23:57:14.395675 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.395530 2562 manager.go:216] Machine: {Timestamp:2026-07-06 23:57:14.394146666 +0000 UTC m=+0.366631319 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100326 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20fe903117dcd5c728ba9e0fff08e8 SystemUUID:ec20fe90-3117-dcd5-c728-ba9e0fff08e8 BootID:b6976b48-44f4-4540-b3b8-356f13783863 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1e:04:69:c0:61 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1e:04:69:c0:61 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:1f:d1:16:5e:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 23:57:14.395675 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.395657 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 23:57:14.395839 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.395755 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 23:57:14.398978 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.398939 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 23:57:14.399191 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.398980 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-52.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 23:57:14.399908 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.399891 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 23:57:14.399908 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.399911 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 23:57:14.400022 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.399925 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:57:14.400646 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.400634 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:57:14.401816 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.401794 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 23:57:14.401894 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.401858 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:57:14.406371 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.406356 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:57:14.406700 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.406689 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 23:57:14.406751 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.406726 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 06 23:57:14.406751 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.406748 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 23:57:14.406827 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.406780 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 23:57:14.406827 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.406792 2562 kubelet.go:398] "Adding apiserver pod source" Jul 06 23:57:14.406827 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.406813 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 23:57:14.407232 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.407220 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:57:14.407277 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.407243 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:57:14.409554 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.409536 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 23:57:14.411562 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.411548 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 23:57:14.411637 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.411579 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 23:57:14.412721 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412710 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 23:57:14.412758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412734 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 23:57:14.412758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412741 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 23:57:14.412758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412748 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 23:57:14.412842 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412761 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 23:57:14.412842 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412770 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 23:57:14.412842 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412781 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 23:57:14.412842 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412786 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 23:57:14.412842 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412796 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 23:57:14.412842 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412802 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 23:57:14.412842 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412816 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 23:57:14.412842 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.412822 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 23:57:14.413604 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.413593 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 23:57:14.413641 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.413607 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 23:57:14.417042 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.417025 2562 server.go:1268] "Started kubelet" Jul 06 23:57:14.417150 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.417098 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 23:57:14.417224 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.417143 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 23:57:14.417264 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.417254 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 23:57:14.418173 ip-10-0-130-52 systemd[1]: Started Kubernetes Kubelet. Jul 06 23:57:14.418754 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.418701 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-47jq5" Jul 06 23:57:14.423085 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.423025 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 23:57:14.423828 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.423781 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 23:57:14.423925 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.423763 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-52.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 23:57:14.424904 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.423970 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-52.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 23:57:14.424904 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.423989 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 06 23:57:14.424904 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.424614 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 23:57:14.427405 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.426466 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-52.ec2.internal.18bfd8b1852ad0c2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-52.ec2.internal,UID:ip-10-0-130-52.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-52.ec2.internal,},FirstTimestamp:2026-07-06 23:57:14.41699245 +0000 UTC m=+0.389477101,LastTimestamp:2026-07-06 23:57:14.41699245 +0000 UTC m=+0.389477101,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-52.ec2.internal,}" Jul 06 23:57:14.427790 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.427767 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-47jq5" Jul 06 23:57:14.428632 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.428618 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 23:57:14.428632 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.428625 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 23:57:14.428900 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.428878 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:14.429521 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.429151 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 23:57:14.429521 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.429174 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 23:57:14.429521 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.429233 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 23:57:14.429521 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.429278 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 06 23:57:14.429521 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.429287 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 06 23:57:14.429802 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.429788 2562 factory.go:55] Registering systemd factory Jul 06 23:57:14.429836 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.429807 2562 factory.go:223] Registration of the systemd container factory successfully Jul 06 23:57:14.431465 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.431449 2562 factory.go:153] Registering CRI-O factory Jul 06 23:57:14.431465 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.431465 2562 factory.go:223] Registration of the crio container factory successfully Jul 06 23:57:14.431586 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.431519 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 23:57:14.431586 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.431539 2562 factory.go:103] Registering Raw factory Jul 06 23:57:14.431586 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.431551 2562 manager.go:1196] Started watching for new ooms in manager Jul 06 23:57:14.431986 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.431973 2562 manager.go:349] Starting recovery of all containers Jul 06 23:57:14.438172 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.438016 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:57:14.438172 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.438043 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 23:57:14.441085 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.441044 2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-52.ec2.internal\" not found" node="ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.442171 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.442150 2562 manager.go:354] Recovery completed Jul 06 23:57:14.447216 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.447199 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:57:14.449424 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.449405 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:57:14.449507 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.449437 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:57:14.449507 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.449448 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:57:14.450037 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.450022 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 23:57:14.450037 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.450035 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 23:57:14.450197 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.450025 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:57:14.450197 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.450054 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:57:14.450834 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.450821 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:57:14.450907 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.450852 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:57:14.450907 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.450866 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:57:14.452175 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.452160 2562 policy_none.go:49] "None policy: Start" Jul 06 23:57:14.452175 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.452176 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 23:57:14.452286 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.452187 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 23:57:14.454054 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.454039 2562 policy_none.go:47] "Start" Jul 06 23:57:14.504881 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.504855 2562 manager.go:337] "Starting Device Plugin manager" Jul 06 23:57:14.505031 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.504917 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 23:57:14.505031 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.504932 2562 server.go:87] "Starting device plugin registration server" Jul 06 23:57:14.505285 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.505272 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 23:57:14.505323 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.505288 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 23:57:14.505411 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.505390 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 23:57:14.505502 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.505493 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 23:57:14.505597 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.505505 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 23:57:14.506079 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.506046 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 23:57:14.506157 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.506109 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:14.576725 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.576692 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 23:57:14.576725 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.576735 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 23:57:14.578713 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.576760 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 23:57:14.578713 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.576794 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 23:57:14.579276 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.579257 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:57:14.606276 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.606182 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:57:14.607414 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.607394 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:57:14.607543 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.607434 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:57:14.607543 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.607451 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:57:14.607543 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.607484 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.613910 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.613888 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.614016 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.613917 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-52.ec2.internal\": node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:14.626926 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.626894 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:14.677822 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.677783 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal"] Jul 06 23:57:14.677955 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.677911 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:57:14.679519 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.679501 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:57:14.679625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.679529 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:57:14.679625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.679539 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:57:14.681713 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.681699 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:57:14.681898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.681881 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.681965 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.681921 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:57:14.682608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.682586 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:57:14.682685 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.682614 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:57:14.682685 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.682636 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:57:14.682685 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.682646 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:57:14.682685 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.682616 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:57:14.682845 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.682699 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:57:14.684934 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.684915 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.685038 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.684943 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:57:14.685802 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.685785 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:57:14.685871 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.685814 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:57:14.685871 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.685825 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:57:14.700762 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.700730 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-52.ec2.internal\" not found" node="ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.704307 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.704286 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-52.ec2.internal\" not found" node="ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.726987 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.726953 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:14.827831 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.827802 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:14.830345 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.830326 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7c5bc94cac8d78d4e207b1bdde47431d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7c5bc94cac8d78d4e207b1bdde47431d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.830412 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.830355 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c5bc94cac8d78d4e207b1bdde47431d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7c5bc94cac8d78d4e207b1bdde47431d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.830412 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.830374 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73905cf8fbe88f74cd23a2f960dd19a6-config\") pod \"kube-apiserver-proxy-ip-10-0-130-52.ec2.internal\" (UID: \"73905cf8fbe88f74cd23a2f960dd19a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.928937 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:14.928871 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:14.931103 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.931081 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73905cf8fbe88f74cd23a2f960dd19a6-config\") pod \"kube-apiserver-proxy-ip-10-0-130-52.ec2.internal\" (UID: \"73905cf8fbe88f74cd23a2f960dd19a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.931179 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.931116 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7c5bc94cac8d78d4e207b1bdde47431d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7c5bc94cac8d78d4e207b1bdde47431d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.931179 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.931134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c5bc94cac8d78d4e207b1bdde47431d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7c5bc94cac8d78d4e207b1bdde47431d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.931272 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.931180 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c5bc94cac8d78d4e207b1bdde47431d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7c5bc94cac8d78d4e207b1bdde47431d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.931272 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.931192 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73905cf8fbe88f74cd23a2f960dd19a6-config\") pod \"kube-apiserver-proxy-ip-10-0-130-52.ec2.internal\" (UID: \"73905cf8fbe88f74cd23a2f960dd19a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" Jul 06 23:57:14.931272 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:14.931186 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7c5bc94cac8d78d4e207b1bdde47431d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7c5bc94cac8d78d4e207b1bdde47431d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" Jul 06 23:57:15.003266 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.003221 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" Jul 06 23:57:15.008264 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.007787 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" Jul 06 23:57:15.029201 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:15.029160 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:15.129598 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:15.129553 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:15.229985 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:15.229901 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:15.330365 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:15.330325 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:15.363680 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.363650 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 23:57:15.364290 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.363838 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 23:57:15.364290 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.363838 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 23:57:15.431130 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:15.431101 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:15.433391 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.433367 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 23:52:14 +0000 UTC" deadline="2027-12-21 00:39:32.841655756 +0000 UTC" Jul 06 23:57:15.433391 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.433389 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12768h42m17.40826995s" Jul 06 23:57:15.531353 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:15.531280 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:15.588443 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:15.588403 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73905cf8fbe88f74cd23a2f960dd19a6.slice/crio-529c3e6226dbd1aa4e7c042a03343866dd14df7a890bb10920faa087850c996d WatchSource:0}: Error finding container 529c3e6226dbd1aa4e7c042a03343866dd14df7a890bb10920faa087850c996d: Status 404 returned error can't find the container with id 529c3e6226dbd1aa4e7c042a03343866dd14df7a890bb10920faa087850c996d Jul 06 23:57:15.588695 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:15.588678 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5bc94cac8d78d4e207b1bdde47431d.slice/crio-6c7ec77398124560e2bbc03d637bf17e738ed3106680381453b8f5e19c37e17b WatchSource:0}: Error finding container 6c7ec77398124560e2bbc03d637bf17e738ed3106680381453b8f5e19c37e17b: Status 404 returned error can't find the container with id 6c7ec77398124560e2bbc03d637bf17e738ed3106680381453b8f5e19c37e17b Jul 06 23:57:15.596803 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.596784 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:57:15.632320 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:15.632276 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:15.732790 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:15.732757 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:15.755365 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.755338 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:57:15.833023 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:15.832946 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found" Jul 06 23:57:15.836627 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.836596 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:57:15.930251 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.930217 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" Jul 06 23:57:15.947463 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.947430 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:57:15.948421 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.948406 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" Jul 06 23:57:15.962995 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:15.962966 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:57:16.244272 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.244183 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:57:16.408259 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.408226 2562 apiserver.go:52] "Watching apiserver" Jul 06 23:57:16.415420 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.415388 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 23:57:16.415852 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.415819 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82","openshift-image-registry/node-ca-bfwh4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal","openshift-multus/network-metrics-daemon-k8l5b","openshift-ovn-kubernetes/ovnkube-node-kr8m4","kube-system/konnectivity-agent-wgncd","openshift-cluster-node-tuning-operator/tuned-zxknp","openshift-dns/node-resolver-d5v6t","openshift-multus/multus-additional-cni-plugins-bvddn","openshift-multus/multus-zfpxw","openshift-network-diagnostics/network-check-target-8ph76","openshift-network-operator/iptables-alerter-zjcqn","kube-system/global-pull-secret-syncer-9vrmj","kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal"] Jul 06 23:57:16.418381 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.418349 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.420652 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.420634 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:16.420753 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:16.420702 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:16.423934 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.423436 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d5v6t" Jul 06 23:57:16.423934 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.423696 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 23:57:16.424115 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.423983 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:57:16.424528 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.424510 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sf2zg\"" Jul 06 23:57:16.425153 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.425133 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:16.425251 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:16.425235 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:16.427904 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.427454 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ztzj5\"" Jul 06 23:57:16.427904 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.427493 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 23:57:16.427904 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.427743 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 23:57:16.428118 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.427918 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.429228 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.429173 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 23:57:16.430227 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.430209 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:16.432976 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.432960 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.433937 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.433918 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 23:57:16.435554 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.435103 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 23:57:16.435554 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.435530 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 23:57:16.435710 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.435579 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8jrcf\"" Jul 06 23:57:16.435710 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.435618 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.436919 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.436680 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 23:57:16.436919 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.436733 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 23:57:16.437175 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.437156 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xs2x2\"" Jul 06 23:57:16.438011 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.437989 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 23:57:16.438135 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.438041 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 23:57:16.438135 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.438045 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bfwh4" Jul 06 23:57:16.438249 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.438219 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 23:57:16.438806 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.438789 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 23:57:16.438891 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.438856 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frnst\" (UniqueName: \"kubernetes.io/projected/43039088-05c7-4888-ac30-35d9d2faee8b-kube-api-access-frnst\") pod \"node-resolver-d5v6t\" (UID: \"43039088-05c7-4888-ac30-35d9d2faee8b\") " pod="openshift-dns/node-resolver-d5v6t" Jul 06 23:57:16.438945 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.438891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e87a7305-1f60-4a49-bff3-2a0082e2c112-konnectivity-ca\") pod \"konnectivity-agent-wgncd\" (UID: \"e87a7305-1f60-4a49-bff3-2a0082e2c112\") " pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:16.438945 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.438924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96p5t\" (UniqueName: \"kubernetes.io/projected/d2f756d2-cf8a-463e-ac13-049452ff8425-kube-api-access-96p5t\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:16.439159 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.438980 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-systemd-units\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.439159 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439011 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-run-netns\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.439159 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-log-socket\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.439328 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439166 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b15cee40-4bc1-4901-8146-553fb2acd3da-ovn-node-metrics-cert\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.439328 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439220 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-modprobe-d\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.439328 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439248 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-var-lib-kubelet\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.439328 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439277 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7888a0e8-ee58-4d66-8720-70feb41fef79-kubelet-config\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:16.439328 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439295 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/43039088-05c7-4888-ac30-35d9d2faee8b-hosts-file\") pod \"node-resolver-d5v6t\" (UID: \"43039088-05c7-4888-ac30-35d9d2faee8b\") " pod="openshift-dns/node-resolver-d5v6t" Jul 06 23:57:16.439328 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439310 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-cni-bin\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.439328 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439328 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.439642 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b15cee40-4bc1-4901-8146-553fb2acd3da-ovnkube-config\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.439642 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439382 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-systemd\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.439642 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439404 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-tmp\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.439642 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439428 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7888a0e8-ee58-4d66-8720-70feb41fef79-dbus\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:16.439642 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439452 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:16.439642 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439478 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:16.439642 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439504 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-var-lib-openvswitch\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.439642 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439529 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-run-ovn-kubernetes\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.439642 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-cni-netd\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.439642 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b15cee40-4bc1-4901-8146-553fb2acd3da-env-overrides\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.440077 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439652 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-sysconfig\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.440077 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439681 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-slash\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.440077 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-etc-openvswitch\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.440077 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439763 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e87a7305-1f60-4a49-bff3-2a0082e2c112-agent-certs\") pod \"konnectivity-agent-wgncd\" (UID: \"e87a7305-1f60-4a49-bff3-2a0082e2c112\") " pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:16.440077 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439795 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-kubernetes\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.440077 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439857 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-run\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.440077 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439903 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-sys\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.440077 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.439965 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-lib-modules\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.440077 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440011 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-run-openvswitch\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.440077 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440020 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b15cee40-4bc1-4901-8146-553fb2acd3da-ovnkube-script-lib\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440138 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-sysctl-d\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-tuned\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440205 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-kubelet\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440258 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-run-systemd\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440286 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-run-ovn\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440331 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-node-log\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440350 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440384 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj844\" (UniqueName: \"kubernetes.io/projected/b15cee40-4bc1-4901-8146-553fb2acd3da-kube-api-access-cj844\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440421 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-sysctl-conf\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-host\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440465 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6sq55\"" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440482 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gssm9\" (UniqueName: \"kubernetes.io/projected/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-kube-api-access-gssm9\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.440608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.440605 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43039088-05c7-4888-ac30-35d9d2faee8b-tmp-dir\") pod \"node-resolver-d5v6t\" (UID: \"43039088-05c7-4888-ac30-35d9d2faee8b\") " pod="openshift-dns/node-resolver-d5v6t" Jul 06 23:57:16.441288 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.441231 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 23:57:16.443207 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.443187 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 23:57:16.443536 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.443525 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 23:57:16.443877 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.443858 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:16.443962 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:16.443921 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:16.444851 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.444505 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6xgt2\"" Jul 06 23:57:16.444851 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.444521 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 23:57:16.444851 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.444568 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 23:57:16.444851 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.444817 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lkwqk\"" Jul 06 23:57:16.445113 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.445030 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 23:57:16.445174 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.445157 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zjcqn" Jul 06 23:57:16.445298 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.445278 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 23:57:16.445472 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.445449 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 23:57:16.445578 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.445564 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 23:57:16.445646 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.445602 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 23:57:16.445707 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.445694 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p9hkp\"" Jul 06 23:57:16.452793 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.452774 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xntdl\"" Jul 06 23:57:16.452886 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.452842 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:57:16.452995 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.452775 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 23:57:16.453189 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.453167 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 23:57:16.473302 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.473278 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:57:16.516932 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.516871 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rbc7r" Jul 06 23:57:16.526726 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.526284 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rbc7r" Jul 06 23:57:16.530430 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.530406 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 23:57:16.540889 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.540852 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-run-ovn\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541060 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.540902 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-node-log\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541060 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.540942 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj844\" (UniqueName: \"kubernetes.io/projected/b15cee40-4bc1-4901-8146-553fb2acd3da-kube-api-access-cj844\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541060 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.540964 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-run-ovn\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541060 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.540972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-sysctl-conf\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.541060 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541022 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-node-log\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541060 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-host\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.541373 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gssm9\" (UniqueName: \"kubernetes.io/projected/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-kube-api-access-gssm9\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.541373 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541110 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxthb\" (UniqueName: \"kubernetes.io/projected/cb806a84-774e-429e-b509-47c704058ae7-kube-api-access-rxthb\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.541373 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541119 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-host\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.541373 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541141 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43039088-05c7-4888-ac30-35d9d2faee8b-tmp-dir\") pod \"node-resolver-d5v6t\" (UID: \"43039088-05c7-4888-ac30-35d9d2faee8b\") " pod="openshift-dns/node-resolver-d5v6t" Jul 06 23:57:16.541373 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541149 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-sysctl-conf\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.541373 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frnst\" (UniqueName: \"kubernetes.io/projected/43039088-05c7-4888-ac30-35d9d2faee8b-kube-api-access-frnst\") pod \"node-resolver-d5v6t\" (UID: \"43039088-05c7-4888-ac30-35d9d2faee8b\") " pod="openshift-dns/node-resolver-d5v6t" Jul 06 23:57:16.541373 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e87a7305-1f60-4a49-bff3-2a0082e2c112-konnectivity-ca\") pod \"konnectivity-agent-wgncd\" (UID: \"e87a7305-1f60-4a49-bff3-2a0082e2c112\") " pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:16.541373 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541319 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-etc-kubernetes\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.541373 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541349 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-systemd-units\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541373 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541374 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-run-netns\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541413 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-log-socket\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541480 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-log-socket\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541492 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-systemd-units\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541513 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b15cee40-4bc1-4901-8146-553fb2acd3da-ovn-node-metrics-cert\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541528 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-run-netns\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541521 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43039088-05c7-4888-ac30-35d9d2faee8b-tmp-dir\") pod \"node-resolver-d5v6t\" (UID: \"43039088-05c7-4888-ac30-35d9d2faee8b\") " pod="openshift-dns/node-resolver-d5v6t" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541579 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8956be1-6671-4c8f-8441-5e5d9c451638-iptables-alerter-script\") pod \"iptables-alerter-zjcqn\" (UID: \"d8956be1-6671-4c8f-8441-5e5d9c451638\") " pod="openshift-network-operator/iptables-alerter-zjcqn" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7888a0e8-ee58-4d66-8720-70feb41fef79-kubelet-config\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541706 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-cni-bin\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541732 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b15cee40-4bc1-4901-8146-553fb2acd3da-ovnkube-config\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541752 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7888a0e8-ee58-4d66-8720-70feb41fef79-kubelet-config\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541758 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541788 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-cni-bin\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541806 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-socket-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541835 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjnq4\" (UniqueName: \"kubernetes.io/projected/354c35b5-4b7a-47c3-825e-ce0decc20f9a-kube-api-access-kjnq4\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541842 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e87a7305-1f60-4a49-bff3-2a0082e2c112-konnectivity-ca\") pod \"konnectivity-agent-wgncd\" (UID: \"e87a7305-1f60-4a49-bff3-2a0082e2c112\") " pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:16.541898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-multus-cni-dir\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541884 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.541889 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-os-release\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542007 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542036 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-var-lib-openvswitch\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-cni-netd\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542100 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-sysconfig\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542138 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-sys\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542158 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-var-lib-openvswitch\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-sys-fs\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542192 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-cni-netd\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:16.542196 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542201 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16c70b55-5ac2-4cd0-8f14-22846a7dbe7a-host\") pod \"node-ca-bfwh4\" (UID: \"16c70b55-5ac2-4cd0-8f14-22846a7dbe7a\") " pod="openshift-image-registry/node-ca-bfwh4" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542161 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-sysconfig\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542219 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-sys\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542225 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-cnibin\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-kubernetes\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.542625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542268 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b15cee40-4bc1-4901-8146-553fb2acd3da-ovnkube-config\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:16.542312 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret podName:7888a0e8-ee58-4d66-8720-70feb41fef79 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:17.042277022 +0000 UTC m=+3.014761669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret") pod "global-pull-secret-syncer-9vrmj" (UID: "7888a0e8-ee58-4d66-8720-70feb41fef79") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542318 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-kubernetes\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542334 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-run\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542362 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-lib-modules\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542365 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-run\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542393 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16c70b55-5ac2-4cd0-8f14-22846a7dbe7a-serviceca\") pod \"node-ca-bfwh4\" (UID: \"16c70b55-5ac2-4cd0-8f14-22846a7dbe7a\") " pod="openshift-image-registry/node-ca-bfwh4" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542418 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-run-k8s-cni-cncf-io\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542444 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-var-lib-cni-bin\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542460 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb806a84-774e-429e-b509-47c704058ae7-multus-daemon-config\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542483 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-run-openvswitch\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542460 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-lib-modules\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542507 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b15cee40-4bc1-4901-8146-553fb2acd3da-ovnkube-script-lib\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-sysctl-d\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542549 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-run-openvswitch\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-tuned\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.543445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542581 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-system-cni-dir\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542598 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-os-release\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542613 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-hostroot\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542634 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5bqb\" (UniqueName: \"kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb\") pod \"network-check-target-8ph76\" (UID: \"9d2f0b67-1dff-4670-8056-4023d2228f8c\") " pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-kubelet\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542666 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-sysctl-d\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8956be1-6671-4c8f-8441-5e5d9c451638-host-slash\") pod \"iptables-alerter-zjcqn\" (UID: \"d8956be1-6671-4c8f-8441-5e5d9c451638\") " pod="openshift-network-operator/iptables-alerter-zjcqn" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542712 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/354c35b5-4b7a-47c3-825e-ce0decc20f9a-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542724 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-kubelet\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542735 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-multus-conf-dir\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542758 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb806a84-774e-429e-b509-47c704058ae7-cni-binary-copy\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542781 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-run-netns\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542802 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-run-multus-certs\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542828 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-registration-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542864 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-cnibin\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542898 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96p5t\" (UniqueName: \"kubernetes.io/projected/d2f756d2-cf8a-463e-ac13-049452ff8425-kube-api-access-96p5t\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-modprobe-d\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.544223 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-var-lib-kubelet\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542989 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b15cee40-4bc1-4901-8146-553fb2acd3da-ovnkube-script-lib\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.542991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-etc-selinux\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543042 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-var-lib-kubelet\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543082 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-modprobe-d\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543086 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/43039088-05c7-4888-ac30-35d9d2faee8b-hosts-file\") pod \"node-resolver-d5v6t\" (UID: \"43039088-05c7-4888-ac30-35d9d2faee8b\") " pod="openshift-dns/node-resolver-d5v6t" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543124 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/43039088-05c7-4888-ac30-35d9d2faee8b-hosts-file\") pod \"node-resolver-d5v6t\" (UID: \"43039088-05c7-4888-ac30-35d9d2faee8b\") " pod="openshift-dns/node-resolver-d5v6t" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543127 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-systemd\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543163 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-tmp\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543201 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-device-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543226 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7888a0e8-ee58-4d66-8720-70feb41fef79-dbus\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543238 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-systemd\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543249 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543271 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-run-ovn-kubernetes\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543293 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b15cee40-4bc1-4901-8146-553fb2acd3da-env-overrides\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.544973 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543316 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:16.543336 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543342 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-system-cni-dir\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543347 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-run-ovn-kubernetes\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-slash\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:16.543417 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs podName:d2f756d2-cf8a-463e-ac13-049452ff8425 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:17.043401201 +0000 UTC m=+3.015885841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs") pod "network-metrics-daemon-k8l5b" (UID: "d2f756d2-cf8a-463e-ac13-049452ff8425") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543415 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7888a0e8-ee58-4d66-8720-70feb41fef79-dbus\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543447 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-etc-openvswitch\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543467 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-host-slash\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543510 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e87a7305-1f60-4a49-bff3-2a0082e2c112-agent-certs\") pod \"konnectivity-agent-wgncd\" (UID: \"e87a7305-1f60-4a49-bff3-2a0082e2c112\") " pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543542 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rrwx\" (UniqueName: \"kubernetes.io/projected/d8956be1-6671-4c8f-8441-5e5d9c451638-kube-api-access-5rrwx\") pod \"iptables-alerter-zjcqn\" (UID: \"d8956be1-6671-4c8f-8441-5e5d9c451638\") " pod="openshift-network-operator/iptables-alerter-zjcqn" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2fk8\" (UniqueName: \"kubernetes.io/projected/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-kube-api-access-z2fk8\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543575 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-etc-openvswitch\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwb9z\" (UniqueName: \"kubernetes.io/projected/16c70b55-5ac2-4cd0-8f14-22846a7dbe7a-kube-api-access-rwb9z\") pod \"node-ca-bfwh4\" (UID: \"16c70b55-5ac2-4cd0-8f14-22846a7dbe7a\") " pod="openshift-image-registry/node-ca-bfwh4" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543695 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b15cee40-4bc1-4901-8146-553fb2acd3da-env-overrides\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543735 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/354c35b5-4b7a-47c3-825e-ce0decc20f9a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.545758 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/354c35b5-4b7a-47c3-825e-ce0decc20f9a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.546247 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-run-systemd\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.546247 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543837 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-multus-socket-dir-parent\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.546247 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-var-lib-cni-multus\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.546247 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-var-lib-kubelet\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.546247 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.543980 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b15cee40-4bc1-4901-8146-553fb2acd3da-run-systemd\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.546247 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.545602 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-tmp\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.546247 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.545645 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-etc-tuned\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.546247 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.545904 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b15cee40-4bc1-4901-8146-553fb2acd3da-ovn-node-metrics-cert\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.546247 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.545980 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e87a7305-1f60-4a49-bff3-2a0082e2c112-agent-certs\") pod \"konnectivity-agent-wgncd\" (UID: \"e87a7305-1f60-4a49-bff3-2a0082e2c112\") " pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:16.565547 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.565520 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:57:16.578292 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.578265 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frnst\" (UniqueName: \"kubernetes.io/projected/43039088-05c7-4888-ac30-35d9d2faee8b-kube-api-access-frnst\") pod \"node-resolver-d5v6t\" (UID: \"43039088-05c7-4888-ac30-35d9d2faee8b\") " pod="openshift-dns/node-resolver-d5v6t" Jul 06 23:57:16.581446 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.581402 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" event={"ID":"7c5bc94cac8d78d4e207b1bdde47431d","Type":"ContainerStarted","Data":"6c7ec77398124560e2bbc03d637bf17e738ed3106680381453b8f5e19c37e17b"} Jul 06 23:57:16.582566 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.582541 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" event={"ID":"73905cf8fbe88f74cd23a2f960dd19a6","Type":"ContainerStarted","Data":"529c3e6226dbd1aa4e7c042a03343866dd14df7a890bb10920faa087850c996d"} Jul 06 23:57:16.589094 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.589024 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gssm9\" (UniqueName: \"kubernetes.io/projected/788f5e8d-a8d5-4af8-ae36-86d5af8127cf-kube-api-access-gssm9\") pod \"tuned-zxknp\" (UID: \"788f5e8d-a8d5-4af8-ae36-86d5af8127cf\") " pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.589242 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.589129 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96p5t\" (UniqueName: \"kubernetes.io/projected/d2f756d2-cf8a-463e-ac13-049452ff8425-kube-api-access-96p5t\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:16.589350 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.589330 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj844\" (UniqueName: \"kubernetes.io/projected/b15cee40-4bc1-4901-8146-553fb2acd3da-kube-api-access-cj844\") pod \"ovnkube-node-kr8m4\" (UID: \"b15cee40-4bc1-4901-8146-553fb2acd3da\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.644305 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644268 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.644492 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644305 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-system-cni-dir\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.644492 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rrwx\" (UniqueName: \"kubernetes.io/projected/d8956be1-6671-4c8f-8441-5e5d9c451638-kube-api-access-5rrwx\") pod \"iptables-alerter-zjcqn\" (UID: \"d8956be1-6671-4c8f-8441-5e5d9c451638\") " pod="openshift-network-operator/iptables-alerter-zjcqn" Jul 06 23:57:16.644492 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2fk8\" (UniqueName: \"kubernetes.io/projected/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-kube-api-access-z2fk8\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.644492 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644412 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwb9z\" (UniqueName: \"kubernetes.io/projected/16c70b55-5ac2-4cd0-8f14-22846a7dbe7a-kube-api-access-rwb9z\") pod \"node-ca-bfwh4\" (UID: \"16c70b55-5ac2-4cd0-8f14-22846a7dbe7a\") " pod="openshift-image-registry/node-ca-bfwh4" Jul 06 23:57:16.644492 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644435 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/354c35b5-4b7a-47c3-825e-ce0decc20f9a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.644492 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644458 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/354c35b5-4b7a-47c3-825e-ce0decc20f9a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.644492 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644465 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-system-cni-dir\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.644492 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644486 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-multus-socket-dir-parent\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.644876 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644546 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-multus-socket-dir-parent\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.644876 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644649 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-var-lib-cni-multus\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.644876 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-var-lib-kubelet\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.644876 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644708 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxthb\" (UniqueName: \"kubernetes.io/projected/cb806a84-774e-429e-b509-47c704058ae7-kube-api-access-rxthb\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.644876 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644760 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-etc-kubernetes\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.644876 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644786 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-var-lib-kubelet\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.644876 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644759 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-var-lib-cni-multus\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.644876 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8956be1-6671-4c8f-8441-5e5d9c451638-iptables-alerter-script\") pod \"iptables-alerter-zjcqn\" (UID: \"d8956be1-6671-4c8f-8441-5e5d9c451638\") " pod="openshift-network-operator/iptables-alerter-zjcqn" Jul 06 23:57:16.644876 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644799 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.644876 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644863 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644893 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-socket-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjnq4\" (UniqueName: \"kubernetes.io/projected/354c35b5-4b7a-47c3-825e-ce0decc20f9a-kube-api-access-kjnq4\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644932 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-multus-cni-dir\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644949 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-os-release\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.644992 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-sys-fs\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645059 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-sys-fs\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645164 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-os-release\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645168 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-multus-cni-dir\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645176 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645177 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/354c35b5-4b7a-47c3-825e-ce0decc20f9a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645203 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/354c35b5-4b7a-47c3-825e-ce0decc20f9a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645244 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-etc-kubernetes\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16c70b55-5ac2-4cd0-8f14-22846a7dbe7a-host\") pod \"node-ca-bfwh4\" (UID: \"16c70b55-5ac2-4cd0-8f14-22846a7dbe7a\") " pod="openshift-image-registry/node-ca-bfwh4" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645298 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-socket-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645304 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8956be1-6671-4c8f-8441-5e5d9c451638-iptables-alerter-script\") pod \"iptables-alerter-zjcqn\" (UID: \"d8956be1-6671-4c8f-8441-5e5d9c451638\") " pod="openshift-network-operator/iptables-alerter-zjcqn" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645302 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-cnibin\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645343 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-cnibin\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.645354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645348 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16c70b55-5ac2-4cd0-8f14-22846a7dbe7a-serviceca\") pod \"node-ca-bfwh4\" (UID: \"16c70b55-5ac2-4cd0-8f14-22846a7dbe7a\") " pod="openshift-image-registry/node-ca-bfwh4" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645366 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16c70b55-5ac2-4cd0-8f14-22846a7dbe7a-host\") pod \"node-ca-bfwh4\" (UID: \"16c70b55-5ac2-4cd0-8f14-22846a7dbe7a\") " pod="openshift-image-registry/node-ca-bfwh4" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-run-k8s-cni-cncf-io\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-var-lib-cni-bin\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645442 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb806a84-774e-429e-b509-47c704058ae7-multus-daemon-config\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645459 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-run-k8s-cni-cncf-io\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645471 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-system-cni-dir\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645482 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-var-lib-cni-bin\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645496 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-os-release\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645527 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-system-cni-dir\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-hostroot\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645548 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-os-release\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645572 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-hostroot\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645595 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bqb\" (UniqueName: \"kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb\") pod \"network-check-target-8ph76\" (UID: \"9d2f0b67-1dff-4670-8056-4023d2228f8c\") " pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645695 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8956be1-6671-4c8f-8441-5e5d9c451638-host-slash\") pod \"iptables-alerter-zjcqn\" (UID: \"d8956be1-6671-4c8f-8441-5e5d9c451638\") " pod="openshift-network-operator/iptables-alerter-zjcqn" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645726 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/354c35b5-4b7a-47c3-825e-ce0decc20f9a-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645737 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8956be1-6671-4c8f-8441-5e5d9c451638-host-slash\") pod \"iptables-alerter-zjcqn\" (UID: \"d8956be1-6671-4c8f-8441-5e5d9c451638\") " pod="openshift-network-operator/iptables-alerter-zjcqn" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645751 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-multus-conf-dir\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646169 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645751 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16c70b55-5ac2-4cd0-8f14-22846a7dbe7a-serviceca\") pod \"node-ca-bfwh4\" (UID: \"16c70b55-5ac2-4cd0-8f14-22846a7dbe7a\") " pod="openshift-image-registry/node-ca-bfwh4" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645804 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-multus-conf-dir\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb806a84-774e-429e-b509-47c704058ae7-cni-binary-copy\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645863 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-run-netns\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-run-multus-certs\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-registration-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645937 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-cnibin\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645941 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-run-netns\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645964 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-etc-selinux\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645995 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb806a84-774e-429e-b509-47c704058ae7-host-run-multus-certs\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.646017 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-registration-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.646000 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/354c35b5-4b7a-47c3-825e-ce0decc20f9a-cnibin\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.645994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-device-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.646086 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-device-dir\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.646098 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb806a84-774e-429e-b509-47c704058ae7-multus-daemon-config\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.646131 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-etc-selinux\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.646224 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/354c35b5-4b7a-47c3-825e-ce0decc20f9a-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.646798 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.646333 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb806a84-774e-429e-b509-47c704058ae7-cni-binary-copy\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.665377 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:16.665340 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:57:16.665377 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:16.665378 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:57:16.665589 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:16.665391 2562 projected.go:196] Error preparing data for projected volume kube-api-access-b5bqb for pod openshift-network-diagnostics/network-check-target-8ph76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:16.665589 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:16.665461 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb podName:9d2f0b67-1dff-4670-8056-4023d2228f8c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:17.165439899 +0000 UTC m=+3.137924543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b5bqb" (UniqueName: "kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb") pod "network-check-target-8ph76" (UID: "9d2f0b67-1dff-4670-8056-4023d2228f8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:16.668422 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.668378 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjnq4\" (UniqueName: \"kubernetes.io/projected/354c35b5-4b7a-47c3-825e-ce0decc20f9a-kube-api-access-kjnq4\") pod \"multus-additional-cni-plugins-bvddn\" (UID: \"354c35b5-4b7a-47c3-825e-ce0decc20f9a\") " pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.670384 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.670360 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rrwx\" (UniqueName: \"kubernetes.io/projected/d8956be1-6671-4c8f-8441-5e5d9c451638-kube-api-access-5rrwx\") pod \"iptables-alerter-zjcqn\" (UID: \"d8956be1-6671-4c8f-8441-5e5d9c451638\") " pod="openshift-network-operator/iptables-alerter-zjcqn" Jul 06 23:57:16.680385 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.680357 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxthb\" (UniqueName: \"kubernetes.io/projected/cb806a84-774e-429e-b509-47c704058ae7-kube-api-access-rxthb\") pod \"multus-zfpxw\" (UID: \"cb806a84-774e-429e-b509-47c704058ae7\") " pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.680609 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.680589 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2fk8\" (UniqueName: \"kubernetes.io/projected/aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23-kube-api-access-z2fk8\") pod \"aws-ebs-csi-driver-node-q4j82\" (UID: \"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.680671 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.680640 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwb9z\" (UniqueName: \"kubernetes.io/projected/16c70b55-5ac2-4cd0-8f14-22846a7dbe7a-kube-api-access-rwb9z\") pod \"node-ca-bfwh4\" (UID: \"16c70b55-5ac2-4cd0-8f14-22846a7dbe7a\") " pod="openshift-image-registry/node-ca-bfwh4" Jul 06 23:57:16.738044 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.738010 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zxknp" Jul 06 23:57:16.748739 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.748711 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d5v6t" Jul 06 23:57:16.758353 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.758290 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:16.765247 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.765220 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:16.774152 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.774119 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bvddn" Jul 06 23:57:16.782128 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.782096 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" Jul 06 23:57:16.789045 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.789018 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bfwh4" Jul 06 23:57:16.798921 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.798895 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zfpxw" Jul 06 23:57:16.806724 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:16.806699 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zjcqn" Jul 06 23:57:17.049230 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.049131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:17.049230 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.049193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:17.049452 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:17.049298 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:17.049452 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:17.049357 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret podName:7888a0e8-ee58-4d66-8720-70feb41fef79 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:18.049342271 +0000 UTC m=+4.021826915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret") pod "global-pull-secret-syncer-9vrmj" (UID: "7888a0e8-ee58-4d66-8720-70feb41fef79") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:17.049452 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:17.049297 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:17.049452 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:17.049436 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs podName:d2f756d2-cf8a-463e-ac13-049452ff8425 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:18.049422841 +0000 UTC m=+4.021907495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs") pod "network-metrics-daemon-k8l5b" (UID: "d2f756d2-cf8a-463e-ac13-049452ff8425") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:17.250790 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.250748 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bqb\" (UniqueName: \"kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb\") pod \"network-check-target-8ph76\" (UID: \"9d2f0b67-1dff-4670-8056-4023d2228f8c\") " pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:17.250974 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:17.250888 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:57:17.250974 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:17.250915 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:57:17.250974 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:17.250929 2562 projected.go:196] Error preparing data for projected volume kube-api-access-b5bqb for pod openshift-network-diagnostics/network-check-target-8ph76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:17.251111 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:17.250983 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb podName:9d2f0b67-1dff-4670-8056-4023d2228f8c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:18.25096779 +0000 UTC m=+4.223452432 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b5bqb" (UniqueName: "kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb") pod "network-check-target-8ph76" (UID: "9d2f0b67-1dff-4670-8056-4023d2228f8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:17.530484 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.530398 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:52:16 +0000 UTC" deadline="2028-04-02 21:09:25.904783218 +0000 UTC" Jul 06 23:57:17.530484 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.530433 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15261h12m8.374352913s" Jul 06 23:57:17.561555 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:17.560957 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa3e0de_fbd1_4fa4_9ab3_17138ac03d23.slice/crio-acd0dc6078c814c03a1b9971a89560991811b4a6ec76c932c6928bc9f8705c6d WatchSource:0}: Error finding container acd0dc6078c814c03a1b9971a89560991811b4a6ec76c932c6928bc9f8705c6d: Status 404 returned error can't find the container with id acd0dc6078c814c03a1b9971a89560991811b4a6ec76c932c6928bc9f8705c6d Jul 06 23:57:17.561555 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:17.561454 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode87a7305_1f60_4a49_bff3_2a0082e2c112.slice/crio-1e01111449de935416e7b499c1306e8048a415a5dd6543848adceda4c563f296 WatchSource:0}: Error finding container 1e01111449de935416e7b499c1306e8048a415a5dd6543848adceda4c563f296: Status 404 returned error can't find the container with id 1e01111449de935416e7b499c1306e8048a415a5dd6543848adceda4c563f296 Jul 06 23:57:17.566840 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:17.566815 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod354c35b5_4b7a_47c3_825e_ce0decc20f9a.slice/crio-d24b0468d5aa97fb85a481f747fdc66593019396eea1f848dd5b4887e6e653c8 WatchSource:0}: Error finding container d24b0468d5aa97fb85a481f747fdc66593019396eea1f848dd5b4887e6e653c8: Status 404 returned error can't find the container with id d24b0468d5aa97fb85a481f747fdc66593019396eea1f848dd5b4887e6e653c8 Jul 06 23:57:17.569616 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:17.569506 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb806a84_774e_429e_b509_47c704058ae7.slice/crio-2448f25276926bf22adf2bc4c709abe0955de3a29d8fb11112c1da1b04f31eb6 WatchSource:0}: Error finding container 2448f25276926bf22adf2bc4c709abe0955de3a29d8fb11112c1da1b04f31eb6: Status 404 returned error can't find the container with id 2448f25276926bf22adf2bc4c709abe0955de3a29d8fb11112c1da1b04f31eb6 Jul 06 23:57:17.570570 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:17.570551 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8956be1_6671_4c8f_8441_5e5d9c451638.slice/crio-afa72cabf1c927af9336dbbecfaf0f7df6ff992322f8d75226658e500e09b686 WatchSource:0}: Error finding container afa72cabf1c927af9336dbbecfaf0f7df6ff992322f8d75226658e500e09b686: Status 404 returned error can't find the container with id afa72cabf1c927af9336dbbecfaf0f7df6ff992322f8d75226658e500e09b686 Jul 06 23:57:17.571586 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:17.571558 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43039088_05c7_4888_ac30_35d9d2faee8b.slice/crio-6aa62e2e7f6392d6fd5718565152fa5107a9b9acd7d9e4b4f5460d7d34e5591c WatchSource:0}: Error finding container 6aa62e2e7f6392d6fd5718565152fa5107a9b9acd7d9e4b4f5460d7d34e5591c: Status 404 returned error can't find the container with id 6aa62e2e7f6392d6fd5718565152fa5107a9b9acd7d9e4b4f5460d7d34e5591c Jul 06 23:57:17.573023 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:17.572998 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16c70b55_5ac2_4cd0_8f14_22846a7dbe7a.slice/crio-be7e7133ee95f06e69cedef1c8d0341ea49c9538ba0fb6231721116b386cbe58 WatchSource:0}: Error finding container be7e7133ee95f06e69cedef1c8d0341ea49c9538ba0fb6231721116b386cbe58: Status 404 returned error can't find the container with id be7e7133ee95f06e69cedef1c8d0341ea49c9538ba0fb6231721116b386cbe58 Jul 06 23:57:17.573429 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:17.573407 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15cee40_4bc1_4901_8146_553fb2acd3da.slice/crio-6119aa82d7b2c8504b81a68ecd7ef07407a01c47154849f85234b9f8ce9059b9 WatchSource:0}: Error finding container 6119aa82d7b2c8504b81a68ecd7ef07407a01c47154849f85234b9f8ce9059b9: Status 404 returned error can't find the container with id 6119aa82d7b2c8504b81a68ecd7ef07407a01c47154849f85234b9f8ce9059b9 Jul 06 23:57:17.574995 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:17.574960 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod788f5e8d_a8d5_4af8_ae36_86d5af8127cf.slice/crio-ffeecd62f568b317e4c66d68449323ab26e95417a072e30b217eb13c37ec3b2d WatchSource:0}: Error finding container ffeecd62f568b317e4c66d68449323ab26e95417a072e30b217eb13c37ec3b2d: Status 404 returned error can't find the container with id ffeecd62f568b317e4c66d68449323ab26e95417a072e30b217eb13c37ec3b2d Jul 06 23:57:17.586225 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.586190 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" event={"ID":"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23","Type":"ContainerStarted","Data":"acd0dc6078c814c03a1b9971a89560991811b4a6ec76c932c6928bc9f8705c6d"} Jul 06 23:57:17.587543 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.587492 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bfwh4" event={"ID":"16c70b55-5ac2-4cd0-8f14-22846a7dbe7a","Type":"ContainerStarted","Data":"be7e7133ee95f06e69cedef1c8d0341ea49c9538ba0fb6231721116b386cbe58"} Jul 06 23:57:17.588795 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.588759 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zfpxw" event={"ID":"cb806a84-774e-429e-b509-47c704058ae7","Type":"ContainerStarted","Data":"2448f25276926bf22adf2bc4c709abe0955de3a29d8fb11112c1da1b04f31eb6"} Jul 06 23:57:17.590000 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.589976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvddn" event={"ID":"354c35b5-4b7a-47c3-825e-ce0decc20f9a","Type":"ContainerStarted","Data":"d24b0468d5aa97fb85a481f747fdc66593019396eea1f848dd5b4887e6e653c8"} Jul 06 23:57:17.591614 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.591589 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zxknp" event={"ID":"788f5e8d-a8d5-4af8-ae36-86d5af8127cf","Type":"ContainerStarted","Data":"ffeecd62f568b317e4c66d68449323ab26e95417a072e30b217eb13c37ec3b2d"} Jul 06 23:57:17.592767 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.592750 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" event={"ID":"b15cee40-4bc1-4901-8146-553fb2acd3da","Type":"ContainerStarted","Data":"6119aa82d7b2c8504b81a68ecd7ef07407a01c47154849f85234b9f8ce9059b9"} Jul 06 23:57:17.593733 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.593715 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d5v6t" event={"ID":"43039088-05c7-4888-ac30-35d9d2faee8b","Type":"ContainerStarted","Data":"6aa62e2e7f6392d6fd5718565152fa5107a9b9acd7d9e4b4f5460d7d34e5591c"} Jul 06 23:57:17.594741 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.594720 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zjcqn" event={"ID":"d8956be1-6671-4c8f-8441-5e5d9c451638","Type":"ContainerStarted","Data":"afa72cabf1c927af9336dbbecfaf0f7df6ff992322f8d75226658e500e09b686"} Jul 06 23:57:17.595672 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:17.595648 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wgncd" event={"ID":"e87a7305-1f60-4a49-bff3-2a0082e2c112","Type":"ContainerStarted","Data":"1e01111449de935416e7b499c1306e8048a415a5dd6543848adceda4c563f296"} Jul 06 23:57:18.058409 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:18.058322 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:18.058584 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:18.058410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:18.058584 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:18.058556 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:18.058693 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:18.058629 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs podName:d2f756d2-cf8a-463e-ac13-049452ff8425 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:20.058606387 +0000 UTC m=+6.031091049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs") pod "network-metrics-daemon-k8l5b" (UID: "d2f756d2-cf8a-463e-ac13-049452ff8425") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:18.059094 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:18.059061 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:18.059170 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:18.059138 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret podName:7888a0e8-ee58-4d66-8720-70feb41fef79 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:20.059120118 +0000 UTC m=+6.031604775 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret") pod "global-pull-secret-syncer-9vrmj" (UID: "7888a0e8-ee58-4d66-8720-70feb41fef79") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:18.260988 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:18.260316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bqb\" (UniqueName: \"kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb\") pod \"network-check-target-8ph76\" (UID: \"9d2f0b67-1dff-4670-8056-4023d2228f8c\") " pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:18.260988 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:18.260510 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:57:18.260988 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:18.260531 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:57:18.260988 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:18.260544 2562 projected.go:196] Error preparing data for projected volume kube-api-access-b5bqb for pod openshift-network-diagnostics/network-check-target-8ph76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:18.260988 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:18.260605 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb podName:9d2f0b67-1dff-4670-8056-4023d2228f8c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:20.260586339 +0000 UTC m=+6.233070996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b5bqb" (UniqueName: "kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb") pod "network-check-target-8ph76" (UID: "9d2f0b67-1dff-4670-8056-4023d2228f8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:18.531613 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:18.531486 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:52:16 +0000 UTC" deadline="2028-01-23 14:31:27.848274992 +0000 UTC" Jul 06 23:57:18.531613 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:18.531527 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13574h34m9.316752707s" Jul 06 23:57:18.580039 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:18.579231 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:18.580039 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:18.579374 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:18.580039 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:18.579833 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:18.580039 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:18.579929 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:18.580739 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:18.580458 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:18.580739 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:18.580570 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:18.617704 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:18.617662 2562 generic.go:356] "Generic (PLEG): container finished" podID="7c5bc94cac8d78d4e207b1bdde47431d" containerID="53f3b2844dec80774eb050e06be5c2bc31dcb745e3656c0f4f406ad9c1d74d15" exitCode=0 Jul 06 23:57:18.617906 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:18.617761 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" event={"ID":"7c5bc94cac8d78d4e207b1bdde47431d","Type":"ContainerDied","Data":"53f3b2844dec80774eb050e06be5c2bc31dcb745e3656c0f4f406ad9c1d74d15"} Jul 06 23:57:18.638743 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:18.638697 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" event={"ID":"73905cf8fbe88f74cd23a2f960dd19a6","Type":"ContainerStarted","Data":"5d428658517388d8a21763293a66e2e8670c377b31dbd0b5c3a743f1a4d0a8fc"} Jul 06 23:57:19.653172 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:19.653133 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" event={"ID":"7c5bc94cac8d78d4e207b1bdde47431d","Type":"ContainerStarted","Data":"554f3caf3e423d72cccbb84d102f7584363792bde8450998756c8323811d3c21"} Jul 06 23:57:19.670543 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:19.669966 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" podStartSLOduration=4.669941346 podStartE2EDuration="4.669941346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:18.657239173 +0000 UTC m=+4.629723834" watchObservedRunningTime="2026-07-06 23:57:19.669941346 +0000 UTC m=+5.642426008" Jul 06 23:57:20.074409 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:20.074314 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:20.074409 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:20.074393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:20.074649 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:20.074534 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:20.074649 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:20.074602 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs podName:d2f756d2-cf8a-463e-ac13-049452ff8425 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:24.074582475 +0000 UTC m=+10.047067130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs") pod "network-metrics-daemon-k8l5b" (UID: "d2f756d2-cf8a-463e-ac13-049452ff8425") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:20.075044 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:20.075022 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:20.075199 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:20.075093 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret podName:7888a0e8-ee58-4d66-8720-70feb41fef79 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:24.075076873 +0000 UTC m=+10.047561525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret") pod "global-pull-secret-syncer-9vrmj" (UID: "7888a0e8-ee58-4d66-8720-70feb41fef79") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:20.276531 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:20.276491 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bqb\" (UniqueName: \"kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb\") pod \"network-check-target-8ph76\" (UID: \"9d2f0b67-1dff-4670-8056-4023d2228f8c\") " pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:20.276722 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:20.276658 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:57:20.276722 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:20.276681 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:57:20.276722 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:20.276693 2562 projected.go:196] Error preparing data for projected volume kube-api-access-b5bqb for pod openshift-network-diagnostics/network-check-target-8ph76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:20.276876 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:20.276755 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb podName:9d2f0b67-1dff-4670-8056-4023d2228f8c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:24.276735287 +0000 UTC m=+10.249219946 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b5bqb" (UniqueName: "kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb") pod "network-check-target-8ph76" (UID: "9d2f0b67-1dff-4670-8056-4023d2228f8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:20.578209 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:20.578179 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:20.578406 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:20.578286 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:20.578406 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:20.578179 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:20.578406 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:20.578354 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:20.579038 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:20.579015 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:20.579187 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:20.579153 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:22.579843 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:22.579759 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:22.580291 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:22.579898 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:22.580291 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:22.580168 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:22.580291 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:22.580279 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:22.580430 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:22.580371 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:22.580430 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:22.580273 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:24.110971 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:24.110927 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:24.111445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:24.111014 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:24.111445 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:24.111121 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:24.111445 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:24.111161 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:24.111445 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:24.111216 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs podName:d2f756d2-cf8a-463e-ac13-049452ff8425 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:32.111184223 +0000 UTC m=+18.083668885 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs") pod "network-metrics-daemon-k8l5b" (UID: "d2f756d2-cf8a-463e-ac13-049452ff8425") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:24.111445 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:24.111238 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret podName:7888a0e8-ee58-4d66-8720-70feb41fef79 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:32.111228039 +0000 UTC m=+18.083712683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret") pod "global-pull-secret-syncer-9vrmj" (UID: "7888a0e8-ee58-4d66-8720-70feb41fef79") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:24.313299 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:24.313255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bqb\" (UniqueName: \"kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb\") pod \"network-check-target-8ph76\" (UID: \"9d2f0b67-1dff-4670-8056-4023d2228f8c\") " pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:24.313490 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:24.313415 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:57:24.313490 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:24.313435 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:57:24.313490 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:24.313445 2562 projected.go:196] Error preparing data for projected volume kube-api-access-b5bqb for pod openshift-network-diagnostics/network-check-target-8ph76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:24.313644 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:24.313500 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb podName:9d2f0b67-1dff-4670-8056-4023d2228f8c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:32.313485705 +0000 UTC m=+18.285970345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b5bqb" (UniqueName: "kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb") pod "network-check-target-8ph76" (UID: "9d2f0b67-1dff-4670-8056-4023d2228f8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:24.578388 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:24.578281 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:24.578556 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:24.578394 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:24.578904 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:24.578883 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:24.579002 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:24.578986 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:24.579141 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:24.579126 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:24.579230 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:24.579212 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:26.577149 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:26.577115 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:26.577611 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:26.577259 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:26.577611 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:26.577473 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:26.577611 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:26.577497 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:26.577611 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:26.577578 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:26.577822 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:26.577669 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:28.577832 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:28.577747 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:28.577832 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:28.577767 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:28.577832 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:28.577776 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:28.578277 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:28.577863 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:28.578277 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:28.577917 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:28.578277 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:28.578013 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:30.578055 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:30.578020 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:30.578479 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:30.578079 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:30.578479 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:30.578031 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:30.578479 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:30.578168 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:30.578479 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:30.578288 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:30.578479 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:30.578369 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:32.172018 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:32.171982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:32.172574 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:32.172122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:32.172574 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:32.172181 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:32.172574 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:32.172253 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs podName:d2f756d2-cf8a-463e-ac13-049452ff8425 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:48.17222949 +0000 UTC m=+34.144714128 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs") pod "network-metrics-daemon-k8l5b" (UID: "d2f756d2-cf8a-463e-ac13-049452ff8425") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:32.172574 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:32.172303 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:32.172574 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:32.172367 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret podName:7888a0e8-ee58-4d66-8720-70feb41fef79 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:48.172348635 +0000 UTC m=+34.144833291 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret") pod "global-pull-secret-syncer-9vrmj" (UID: "7888a0e8-ee58-4d66-8720-70feb41fef79") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:32.373005 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:32.372971 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bqb\" (UniqueName: \"kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb\") pod \"network-check-target-8ph76\" (UID: \"9d2f0b67-1dff-4670-8056-4023d2228f8c\") " pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:32.373160 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:32.373124 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:57:32.373160 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:32.373138 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:57:32.373160 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:32.373146 2562 projected.go:196] Error preparing data for projected volume kube-api-access-b5bqb for pod openshift-network-diagnostics/network-check-target-8ph76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:32.373254 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:32.373193 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb podName:9d2f0b67-1dff-4670-8056-4023d2228f8c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:48.373176882 +0000 UTC m=+34.345661528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b5bqb" (UniqueName: "kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb") pod "network-check-target-8ph76" (UID: "9d2f0b67-1dff-4670-8056-4023d2228f8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:32.578005 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:32.577924 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:32.578005 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:32.577942 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:32.578238 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:32.577929 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:32.578238 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:32.578037 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:32.578238 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:32.578149 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:32.578366 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:32.578239 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:34.578095 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:34.577994 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:34.578553 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:34.578119 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:34.578553 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:34.578169 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:34.578553 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:34.578232 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:34.578553 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:34.578345 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:34.578553 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:34.578421 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:36.577666 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:36.577636 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:36.578034 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:36.577636 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:36.578034 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:36.577767 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:36.578034 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:36.577786 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:36.578034 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:36.577941 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:36.578034 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:36.577971 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:37.689703 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.689336 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zxknp" event={"ID":"788f5e8d-a8d5-4af8-ae36-86d5af8127cf","Type":"ContainerStarted","Data":"8e5a12c59a7ed4b6f9d80a3d740196219cdefe8ef6054ec56a816437afae6e58"} Jul 06 23:57:37.692626 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.692597 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" event={"ID":"b15cee40-4bc1-4901-8146-553fb2acd3da","Type":"ContainerStarted","Data":"f66dc277d8fdb95d36a6f0d3738b8dc8fb39de25a6d2a5a18e63bc421782e8a8"} Jul 06 23:57:37.692755 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.692662 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" event={"ID":"b15cee40-4bc1-4901-8146-553fb2acd3da","Type":"ContainerStarted","Data":"18fbe1ee0ea03fb127444074480584ea7b6503ccedf124cfad018b4213019648"} Jul 06 23:57:37.692755 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.692692 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" event={"ID":"b15cee40-4bc1-4901-8146-553fb2acd3da","Type":"ContainerStarted","Data":"7ab2fd939b2e03fae0cfdb13c59c12c5a33282967df71a1af898e6d42747c746"} Jul 06 23:57:37.692755 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.692736 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" event={"ID":"b15cee40-4bc1-4901-8146-553fb2acd3da","Type":"ContainerStarted","Data":"4e1134ac9b47e6d57088a571d42c4e97b4940c4156e88bbccff784f9a3381702"} Jul 06 23:57:37.692887 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.692782 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" event={"ID":"b15cee40-4bc1-4901-8146-553fb2acd3da","Type":"ContainerStarted","Data":"45fce9a568540c0381b97e1744b5e3ad13de9972eed87416e84e0613e645a709"} Jul 06 23:57:37.692887 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.692818 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" event={"ID":"b15cee40-4bc1-4901-8146-553fb2acd3da","Type":"ContainerStarted","Data":"ea30355aea039bb28d83efc1749ad2b88e624e95f536ca1d61aae7ab978d1566"} Jul 06 23:57:37.694117 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.694093 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d5v6t" event={"ID":"43039088-05c7-4888-ac30-35d9d2faee8b","Type":"ContainerStarted","Data":"b0d5b2d63801fbe67cbfa5e2c0efb8651eeb5cf86426dc797bb5455e968aa6ce"} Jul 06 23:57:37.695638 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.695611 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wgncd" event={"ID":"e87a7305-1f60-4a49-bff3-2a0082e2c112","Type":"ContainerStarted","Data":"686358cd2eba2e7d398a626705786328204d04d0810bec2877b8d4b35938fbd1"} Jul 06 23:57:37.697211 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.697181 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" event={"ID":"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23","Type":"ContainerStarted","Data":"690f19140705406fc9e9fd6fade10ee529bb92b23d98ad4950e882d878aa83a9"} Jul 06 23:57:37.698652 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.698627 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bfwh4" event={"ID":"16c70b55-5ac2-4cd0-8f14-22846a7dbe7a","Type":"ContainerStarted","Data":"96079108a7c9e61f097ee0382568076d5421fc2cf9089059b92bf61563e6034a"} Jul 06 23:57:37.700173 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.700148 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zfpxw" event={"ID":"cb806a84-774e-429e-b509-47c704058ae7","Type":"ContainerStarted","Data":"e9d0789e7ddd6fb92ccbc15de1d4a92fc8bb85c67b545c95b0dd18c3a853911d"} Jul 06 23:57:37.701797 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.701773 2562 generic.go:356] "Generic (PLEG): container finished" podID="354c35b5-4b7a-47c3-825e-ce0decc20f9a" containerID="d8d05468d69deca4a2f6ddab35d9ab22f627b9cfdc0a86086c09cdd5d5b88d1d" exitCode=0 Jul 06 23:57:37.701898 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.701809 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvddn" event={"ID":"354c35b5-4b7a-47c3-825e-ce0decc20f9a","Type":"ContainerDied","Data":"d8d05468d69deca4a2f6ddab35d9ab22f627b9cfdc0a86086c09cdd5d5b88d1d"} Jul 06 23:57:37.722121 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.722055 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" podStartSLOduration=22.722037635 podStartE2EDuration="22.722037635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:19.670511441 +0000 UTC m=+5.642996104" watchObservedRunningTime="2026-07-06 23:57:37.722037635 +0000 UTC m=+23.694522297" Jul 06 23:57:37.722471 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.722445 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zxknp" podStartSLOduration=4.696311651 podStartE2EDuration="23.722436194s" totalImagesPullingTime="19.026124543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:37.721630444 +0000 UTC m=+23.694115105" watchObservedRunningTime="2026-07-06 23:57:37.722436194 +0000 UTC m=+23.694920855" Jul 06 23:57:37.833528 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.833343 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wgncd" podStartSLOduration=5.247268364 podStartE2EDuration="23.833326787s" totalImagesPullingTime="18.586058423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:37.77594473 +0000 UTC m=+23.748429392" watchObservedRunningTime="2026-07-06 23:57:37.833326787 +0000 UTC m=+23.805811447" Jul 06 23:57:37.906490 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.906414 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bfwh4" podStartSLOduration=4.88078796 podStartE2EDuration="23.906398588s" totalImagesPullingTime="19.025610628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:37.906291949 +0000 UTC m=+23.878776610" watchObservedRunningTime="2026-07-06 23:57:37.906398588 +0000 UTC m=+23.878883251" Jul 06 23:57:37.906846 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.906823 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zfpxw" podStartSLOduration=4.617989121 podStartE2EDuration="23.906817326s" totalImagesPullingTime="19.288828205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:37.874401876 +0000 UTC m=+23.846886536" watchObservedRunningTime="2026-07-06 23:57:37.906817326 +0000 UTC m=+23.879301986" Jul 06 23:57:37.936047 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:37.936005 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-d5v6t" podStartSLOduration=4.902057468 podStartE2EDuration="23.935991792s" totalImagesPullingTime="19.033934324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:37.935339234 +0000 UTC m=+23.907823892" watchObservedRunningTime="2026-07-06 23:57:37.935991792 +0000 UTC m=+23.908476454" Jul 06 23:57:38.316436 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:38.316413 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 23:57:38.519788 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:38.519640 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T23:57:38.316430778Z","UUID":"b035a14c-3b10-45f9-8cfd-a575342e961c","Handler":null,"Name":"","Endpoint":""} Jul 06 23:57:38.521423 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:38.521400 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 23:57:38.521423 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:38.521440 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 23:57:38.577934 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:38.577906 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:38.578122 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:38.577906 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:38.578122 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:38.578022 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:38.578245 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:38.578164 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:38.578245 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:38.578236 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:38.578346 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:38.578318 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:38.706852 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:38.706811 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zjcqn" event={"ID":"d8956be1-6671-4c8f-8441-5e5d9c451638","Type":"ContainerStarted","Data":"bbe6906ead4ccd89e9f99664b5a2034cbfbe75cb4129b5bb4b61357760bce9c8"} Jul 06 23:57:38.713119 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:38.713058 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" event={"ID":"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23","Type":"ContainerStarted","Data":"72610f79b68f018bb5bc8ef2f7058851222596f42aa600d27303e1353230cae4"} Jul 06 23:57:38.724243 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:38.724199 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zjcqn" podStartSLOduration=5.725336165 podStartE2EDuration="24.724185019s" totalImagesPullingTime="18.998848854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:38.723627559 +0000 UTC m=+24.696112217" watchObservedRunningTime="2026-07-06 23:57:38.724185019 +0000 UTC m=+24.696669679" Jul 06 23:57:39.717508 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:39.717469 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" event={"ID":"aaa3e0de-fbd1-4fa4-9ab3-17138ac03d23","Type":"ContainerStarted","Data":"5ccba0d47945475254e97bc4d8a4598cf1fae5922f0b9024dc1949a979b4c142"} Jul 06 23:57:39.721448 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:39.721413 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" event={"ID":"b15cee40-4bc1-4901-8146-553fb2acd3da","Type":"ContainerStarted","Data":"37e8ae0cc16466e1e3f04ac1e423d78bb53b3dd71f269c1e3c28869ce53e16ce"} Jul 06 23:57:39.738656 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:39.738610 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q4j82" podStartSLOduration=4.513697597 podStartE2EDuration="25.738591392s" totalImagesPullingTime="21.224893795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:39.738485148 +0000 UTC m=+25.710969809" watchObservedRunningTime="2026-07-06 23:57:39.738591392 +0000 UTC m=+25.711076053" Jul 06 23:57:40.577693 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:40.577646 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:40.577891 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:40.577646 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:40.577891 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:40.577787 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:40.577891 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:40.577859 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:40.577891 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:40.577646 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:40.578110 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:40.577921 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:41.901133 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:41.900937 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:41.901704 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:41.901656 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:42.577944 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:42.577910 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:42.578181 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:42.577915 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:42.578181 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:42.578034 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:42.578181 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:42.578078 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:42.578181 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:42.578173 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:42.578333 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:42.578253 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:42.730982 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:42.730944 2562 generic.go:356] "Generic (PLEG): container finished" podID="354c35b5-4b7a-47c3-825e-ce0decc20f9a" containerID="4136f41b2bb217ae98e7a07b180e76d876a029ae54e6798410b923f229312b4c" exitCode=0 Jul 06 23:57:42.731145 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:42.730982 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvddn" event={"ID":"354c35b5-4b7a-47c3-825e-ce0decc20f9a","Type":"ContainerDied","Data":"4136f41b2bb217ae98e7a07b180e76d876a029ae54e6798410b923f229312b4c"} Jul 06 23:57:42.735020 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:42.734985 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" event={"ID":"b15cee40-4bc1-4901-8146-553fb2acd3da","Type":"ContainerStarted","Data":"afaaa1a984310f55a721ad6a7c99fd7be03f104cea7e33d4919b99d8c6d71f7f"} Jul 06 23:57:42.811551 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:42.811499 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" podStartSLOduration=9.783823707 podStartE2EDuration="28.811483598s" totalImagesPullingTime="19.027659891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:42.811460988 +0000 UTC m=+28.783945671" watchObservedRunningTime="2026-07-06 23:57:42.811483598 +0000 UTC m=+28.783968259" Jul 06 23:57:43.737596 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:43.737565 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:43.738161 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:43.737647 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:43.738161 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:43.737684 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:43.752490 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:43.752463 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:43.752651 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:43.752616 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:57:44.062475 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:44.062097 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8ph76"] Jul 06 23:57:44.062611 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:44.062479 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:44.062611 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:44.062588 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:44.062826 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:44.062804 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9vrmj"] Jul 06 23:57:44.062937 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:44.062925 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:44.063042 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:44.063021 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:44.064541 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:44.064517 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k8l5b"] Jul 06 23:57:44.064651 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:44.064618 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:44.064713 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:44.064697 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:44.741665 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:44.741632 2562 generic.go:356] "Generic (PLEG): container finished" podID="354c35b5-4b7a-47c3-825e-ce0decc20f9a" containerID="94caf3caad41586417142c74ab4eab208881d7310d4251b4e8e153371c715f14" exitCode=0 Jul 06 23:57:44.742382 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:44.741703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvddn" event={"ID":"354c35b5-4b7a-47c3-825e-ce0decc20f9a","Type":"ContainerDied","Data":"94caf3caad41586417142c74ab4eab208881d7310d4251b4e8e153371c715f14"} Jul 06 23:57:45.263569 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:45.263536 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:45.263791 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:45.263691 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 23:57:45.264162 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:45.264144 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wgncd" Jul 06 23:57:45.577500 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:45.577409 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:45.577500 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:45.577452 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:45.577725 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:45.577509 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:45.577725 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:45.577526 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:45.577725 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:45.577590 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:45.577725 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:45.577671 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:46.748665 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:46.748628 2562 generic.go:356] "Generic (PLEG): container finished" podID="354c35b5-4b7a-47c3-825e-ce0decc20f9a" containerID="af670192676068273c5bdc5172a0cd1b1ba0f2c24c369f8c62e716fc51ac7fdf" exitCode=0 Jul 06 23:57:46.749238 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:46.748688 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvddn" event={"ID":"354c35b5-4b7a-47c3-825e-ce0decc20f9a","Type":"ContainerDied","Data":"af670192676068273c5bdc5172a0cd1b1ba0f2c24c369f8c62e716fc51ac7fdf"} Jul 06 23:57:47.577400 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:47.577366 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:47.577603 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:47.577365 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:47.577603 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:47.577507 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:47.577603 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:47.577500 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:47.577745 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:47.577668 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:47.577788 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:47.577752 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:48.187254 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:48.187217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:48.187737 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:48.187280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:48.187737 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:48.187398 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:48.187737 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:48.187402 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:48.187737 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:48.187475 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret podName:7888a0e8-ee58-4d66-8720-70feb41fef79 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:20.187442642 +0000 UTC m=+66.159927283 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret") pod "global-pull-secret-syncer-9vrmj" (UID: "7888a0e8-ee58-4d66-8720-70feb41fef79") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:57:48.187737 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:48.187491 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs podName:d2f756d2-cf8a-463e-ac13-049452ff8425 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:20.187484288 +0000 UTC m=+66.159968927 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs") pod "network-metrics-daemon-k8l5b" (UID: "d2f756d2-cf8a-463e-ac13-049452ff8425") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:57:48.388670 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:48.388636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bqb\" (UniqueName: \"kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb\") pod \"network-check-target-8ph76\" (UID: \"9d2f0b67-1dff-4670-8056-4023d2228f8c\") " pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:48.388871 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:48.388761 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:57:48.388871 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:48.388779 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:57:48.388871 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:48.388793 2562 projected.go:196] Error preparing data for projected volume kube-api-access-b5bqb for pod openshift-network-diagnostics/network-check-target-8ph76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:48.388871 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:48.388854 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb podName:9d2f0b67-1dff-4670-8056-4023d2228f8c nodeName:}" failed. No retries permitted until 2026-07-06 23:58:20.388840656 +0000 UTC m=+66.361325309 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b5bqb" (UniqueName: "kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb") pod "network-check-target-8ph76" (UID: "9d2f0b67-1dff-4670-8056-4023d2228f8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:57:49.577315 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:49.577096 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:49.577896 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:49.577097 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:49.577896 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:49.577116 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:49.577896 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:49.577425 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8l5b" podUID="d2f756d2-cf8a-463e-ac13-049452ff8425" Jul 06 23:57:49.577896 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:49.577510 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ph76" podUID="9d2f0b67-1dff-4670-8056-4023d2228f8c" Jul 06 23:57:49.577896 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:49.577622 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vrmj" podUID="7888a0e8-ee58-4d66-8720-70feb41fef79" Jul 06 23:57:50.773621 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.773546 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeReady" Jul 06 23:57:50.773966 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.773688 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 23:57:50.822363 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.822327 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dprmh"] Jul 06 23:57:50.834854 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.834825 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7xhjt"] Jul 06 23:57:50.835025 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.834973 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dprmh" Jul 06 23:57:50.838495 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.838411 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 23:57:50.839813 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.839788 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 23:57:50.839918 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.839841 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2f46f\"" Jul 06 23:57:50.840594 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.840573 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 23:57:50.853112 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.853091 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dprmh"] Jul 06 23:57:50.853206 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.853126 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xhjt"] Jul 06 23:57:50.853206 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.853171 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:50.855890 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.855876 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s7rq7\"" Jul 06 23:57:50.859243 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.859222 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 23:57:50.859329 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.859312 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 23:57:50.872757 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.872735 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tr5kc"] Jul 06 23:57:50.891682 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.891659 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:50.905924 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.905901 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d914f67-2aac-4cca-91ea-77dd7f0eca61-cert\") pod \"ingress-canary-dprmh\" (UID: \"9d914f67-2aac-4cca-91ea-77dd7f0eca61\") " pod="openshift-ingress-canary/ingress-canary-dprmh" Jul 06 23:57:50.905924 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.905932 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jbfh\" (UniqueName: \"kubernetes.io/projected/9d914f67-2aac-4cca-91ea-77dd7f0eca61-kube-api-access-2jbfh\") pod \"ingress-canary-dprmh\" (UID: \"9d914f67-2aac-4cca-91ea-77dd7f0eca61\") " pod="openshift-ingress-canary/ingress-canary-dprmh" Jul 06 23:57:50.908680 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.908660 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 23:57:50.908799 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.908783 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 23:57:50.908848 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.908827 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 23:57:50.908896 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.908853 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 23:57:50.908896 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.908864 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9lqq6\"" Jul 06 23:57:50.910382 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:50.910366 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tr5kc"] Jul 06 23:57:51.007101 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.007043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed424306-bd32-44c0-ac6b-354738a1eedd-tmp-dir\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.007101 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.007088 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/05598b83-7ee6-42f2-977c-a41b3738a4b7-data-volume\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.007289 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.007111 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.007289 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.007137 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.007289 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.007167 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2zwq\" (UniqueName: \"kubernetes.io/projected/ed424306-bd32-44c0-ac6b-354738a1eedd-kube-api-access-r2zwq\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.007289 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.007191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmgss\" (UniqueName: \"kubernetes.io/projected/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-api-access-rmgss\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.007413 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.007315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d914f67-2aac-4cca-91ea-77dd7f0eca61-cert\") pod \"ingress-canary-dprmh\" (UID: \"9d914f67-2aac-4cca-91ea-77dd7f0eca61\") " pod="openshift-ingress-canary/ingress-canary-dprmh" Jul 06 23:57:51.007413 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.007342 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jbfh\" (UniqueName: \"kubernetes.io/projected/9d914f67-2aac-4cca-91ea-77dd7f0eca61-kube-api-access-2jbfh\") pod \"ingress-canary-dprmh\" (UID: \"9d914f67-2aac-4cca-91ea-77dd7f0eca61\") " pod="openshift-ingress-canary/ingress-canary-dprmh" Jul 06 23:57:51.007413 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.007361 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/05598b83-7ee6-42f2-977c-a41b3738a4b7-crio-socket\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.007413 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.007381 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.007526 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.007433 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ed424306-bd32-44c0-ac6b-354738a1eedd-config-volume\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.011636 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.011614 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d914f67-2aac-4cca-91ea-77dd7f0eca61-cert\") pod \"ingress-canary-dprmh\" (UID: \"9d914f67-2aac-4cca-91ea-77dd7f0eca61\") " pod="openshift-ingress-canary/ingress-canary-dprmh" Jul 06 23:57:51.018968 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.018942 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jbfh\" (UniqueName: \"kubernetes.io/projected/9d914f67-2aac-4cca-91ea-77dd7f0eca61-kube-api-access-2jbfh\") pod \"ingress-canary-dprmh\" (UID: \"9d914f67-2aac-4cca-91ea-77dd7f0eca61\") " pod="openshift-ingress-canary/ingress-canary-dprmh" Jul 06 23:57:51.107722 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.107693 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed424306-bd32-44c0-ac6b-354738a1eedd-tmp-dir\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.107858 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.107726 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/05598b83-7ee6-42f2-977c-a41b3738a4b7-data-volume\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.107858 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.107749 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.107858 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.107773 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.107858 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.107802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2zwq\" (UniqueName: \"kubernetes.io/projected/ed424306-bd32-44c0-ac6b-354738a1eedd-kube-api-access-r2zwq\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.107858 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.107831 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmgss\" (UniqueName: \"kubernetes.io/projected/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-api-access-rmgss\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.107858 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.107848 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:57:51.108187 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.107896 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:57:51.108187 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.107917 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm podName:05598b83-7ee6-42f2-977c-a41b3738a4b7 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:51.607899379 +0000 UTC m=+37.580384041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tr5kc" (UID: "05598b83-7ee6-42f2-977c-a41b3738a4b7") : configmap "kube-rbac-proxy" not found Jul 06 23:57:51.108187 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.107965 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls podName:ed424306-bd32-44c0-ac6b-354738a1eedd nodeName:}" failed. No retries permitted until 2026-07-06 23:57:51.607943652 +0000 UTC m=+37.580428310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls") pod "dns-default-7xhjt" (UID: "ed424306-bd32-44c0-ac6b-354738a1eedd") : secret "dns-default-metrics-tls" not found Jul 06 23:57:51.108187 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.107903 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/05598b83-7ee6-42f2-977c-a41b3738a4b7-crio-socket\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.108187 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.108011 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.108187 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.108084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ed424306-bd32-44c0-ac6b-354738a1eedd-config-volume\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.108187 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.108097 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/05598b83-7ee6-42f2-977c-a41b3738a4b7-crio-socket\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.108187 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.108116 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/05598b83-7ee6-42f2-977c-a41b3738a4b7-data-volume\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.108187 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.108139 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:57:51.108187 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.108137 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed424306-bd32-44c0-ac6b-354738a1eedd-tmp-dir\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.108187 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.108194 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls podName:05598b83-7ee6-42f2-977c-a41b3738a4b7 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:51.608180418 +0000 UTC m=+37.580665074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tr5kc" (UID: "05598b83-7ee6-42f2-977c-a41b3738a4b7") : secret "insights-runtime-extractor-tls" not found Jul 06 23:57:51.108542 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.108526 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ed424306-bd32-44c0-ac6b-354738a1eedd-config-volume\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.116713 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.116692 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2zwq\" (UniqueName: \"kubernetes.io/projected/ed424306-bd32-44c0-ac6b-354738a1eedd-kube-api-access-r2zwq\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.116796 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.116752 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmgss\" (UniqueName: \"kubernetes.io/projected/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-api-access-rmgss\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.145917 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.145891 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dprmh" Jul 06 23:57:51.343931 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.343839 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dprmh"] Jul 06 23:57:51.363056 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:51.363018 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d914f67_2aac_4cca_91ea_77dd7f0eca61.slice/crio-fb0052f34fbd98482c8fa3ce8e6d97cd8058e9e4cd091ec4f5cb356cdd3a717c WatchSource:0}: Error finding container fb0052f34fbd98482c8fa3ce8e6d97cd8058e9e4cd091ec4f5cb356cdd3a717c: Status 404 returned error can't find the container with id fb0052f34fbd98482c8fa3ce8e6d97cd8058e9e4cd091ec4f5cb356cdd3a717c Jul 06 23:57:51.514820 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.514787 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k"] Jul 06 23:57:51.516445 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.516430 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k" Jul 06 23:57:51.518955 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.518932 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 23:57:51.518955 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.518932 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 23:57:51.519134 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.518994 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ndb8z\"" Jul 06 23:57:51.527359 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.527337 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k"] Jul 06 23:57:51.578044 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.578011 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:57:51.578244 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.578135 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:57:51.578311 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.578284 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:57:51.581900 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.581868 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:57:51.582042 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.581908 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wdx8b\"" Jul 06 23:57:51.582248 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.582226 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:57:51.582348 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.582273 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:57:51.582405 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.582353 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:57:51.582405 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.582388 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v6xkb\"" Jul 06 23:57:51.611814 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.611787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv8xt\" (UniqueName: \"kubernetes.io/projected/fe3dbf5d-804e-4447-8bec-408cfe773dbf-kube-api-access-mv8xt\") pod \"migrator-d57d58cc9-bn52k\" (UID: \"fe3dbf5d-804e-4447-8bec-408cfe773dbf\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k" Jul 06 23:57:51.611960 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.611823 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.611960 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.611862 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:51.611960 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.611916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:51.611960 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.611929 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:57:51.612172 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.611968 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:57:51.612172 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.611976 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm podName:05598b83-7ee6-42f2-977c-a41b3738a4b7 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:52.611962133 +0000 UTC m=+38.584446803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tr5kc" (UID: "05598b83-7ee6-42f2-977c-a41b3738a4b7") : configmap "kube-rbac-proxy" not found Jul 06 23:57:51.612172 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.612038 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls podName:05598b83-7ee6-42f2-977c-a41b3738a4b7 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:52.612019157 +0000 UTC m=+38.584503803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tr5kc" (UID: "05598b83-7ee6-42f2-977c-a41b3738a4b7") : secret "insights-runtime-extractor-tls" not found Jul 06 23:57:51.612172 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.612010 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:57:51.612172 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:51.612103 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls podName:ed424306-bd32-44c0-ac6b-354738a1eedd nodeName:}" failed. No retries permitted until 2026-07-06 23:57:52.612089803 +0000 UTC m=+38.584574449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls") pod "dns-default-7xhjt" (UID: "ed424306-bd32-44c0-ac6b-354738a1eedd") : secret "dns-default-metrics-tls" not found Jul 06 23:57:51.712934 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.712895 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv8xt\" (UniqueName: \"kubernetes.io/projected/fe3dbf5d-804e-4447-8bec-408cfe773dbf-kube-api-access-mv8xt\") pod \"migrator-d57d58cc9-bn52k\" (UID: \"fe3dbf5d-804e-4447-8bec-408cfe773dbf\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k" Jul 06 23:57:51.727214 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.727189 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv8xt\" (UniqueName: \"kubernetes.io/projected/fe3dbf5d-804e-4447-8bec-408cfe773dbf-kube-api-access-mv8xt\") pod \"migrator-d57d58cc9-bn52k\" (UID: \"fe3dbf5d-804e-4447-8bec-408cfe773dbf\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k" Jul 06 23:57:51.764990 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.764949 2562 generic.go:356] "Generic (PLEG): container finished" podID="354c35b5-4b7a-47c3-825e-ce0decc20f9a" containerID="9448a64714b8004a23d3f4566a832fd7534a889919526aca773b09c543134edc" exitCode=0 Jul 06 23:57:51.765182 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.765004 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvddn" event={"ID":"354c35b5-4b7a-47c3-825e-ce0decc20f9a","Type":"ContainerDied","Data":"9448a64714b8004a23d3f4566a832fd7534a889919526aca773b09c543134edc"} Jul 06 23:57:51.766329 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.766303 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dprmh" event={"ID":"9d914f67-2aac-4cca-91ea-77dd7f0eca61","Type":"ContainerStarted","Data":"fb0052f34fbd98482c8fa3ce8e6d97cd8058e9e4cd091ec4f5cb356cdd3a717c"} Jul 06 23:57:51.825405 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.825208 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k" Jul 06 23:57:51.978539 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:51.978506 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k"] Jul 06 23:57:51.982538 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:51.982495 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe3dbf5d_804e_4447_8bec_408cfe773dbf.slice/crio-2568e542e704eb79ef58d1073fc2f82ace4cb792d2af3b9b92be128122c0384a WatchSource:0}: Error finding container 2568e542e704eb79ef58d1073fc2f82ace4cb792d2af3b9b92be128122c0384a: Status 404 returned error can't find the container with id 2568e542e704eb79ef58d1073fc2f82ace4cb792d2af3b9b92be128122c0384a Jul 06 23:57:52.348517 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:52.348439 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d5v6t_43039088-05c7-4888-ac30-35d9d2faee8b/dns-node-resolver/0.log" Jul 06 23:57:52.620691 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:52.620655 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:52.620887 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:52.620699 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:52.620887 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:52.620798 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:52.620887 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:52.620817 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:57:52.621045 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:52.620896 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm podName:05598b83-7ee6-42f2-977c-a41b3738a4b7 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:54.620874764 +0000 UTC m=+40.593359412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tr5kc" (UID: "05598b83-7ee6-42f2-977c-a41b3738a4b7") : configmap "kube-rbac-proxy" not found Jul 06 23:57:52.621045 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:52.620915 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:57:52.621045 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:52.620921 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:57:52.621045 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:52.620956 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls podName:05598b83-7ee6-42f2-977c-a41b3738a4b7 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:54.620944243 +0000 UTC m=+40.593428882 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tr5kc" (UID: "05598b83-7ee6-42f2-977c-a41b3738a4b7") : secret "insights-runtime-extractor-tls" not found Jul 06 23:57:52.621045 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:52.620979 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls podName:ed424306-bd32-44c0-ac6b-354738a1eedd nodeName:}" failed. No retries permitted until 2026-07-06 23:57:54.620960354 +0000 UTC m=+40.593444995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls") pod "dns-default-7xhjt" (UID: "ed424306-bd32-44c0-ac6b-354738a1eedd") : secret "dns-default-metrics-tls" not found Jul 06 23:57:52.774603 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:52.774562 2562 generic.go:356] "Generic (PLEG): container finished" podID="354c35b5-4b7a-47c3-825e-ce0decc20f9a" containerID="44abad551a7ccf2e93b696ccc22f99b0ee71c65a309133143c5022aee3fd9777" exitCode=0 Jul 06 23:57:52.774779 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:52.774661 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvddn" event={"ID":"354c35b5-4b7a-47c3-825e-ce0decc20f9a","Type":"ContainerDied","Data":"44abad551a7ccf2e93b696ccc22f99b0ee71c65a309133143c5022aee3fd9777"} Jul 06 23:57:52.776292 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:52.776224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k" event={"ID":"fe3dbf5d-804e-4447-8bec-408cfe773dbf","Type":"ContainerStarted","Data":"2568e542e704eb79ef58d1073fc2f82ace4cb792d2af3b9b92be128122c0384a"} Jul 06 23:57:53.328287 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:53.328254 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bfwh4_16c70b55-5ac2-4cd0-8f14-22846a7dbe7a/node-ca/0.log" Jul 06 23:57:54.638446 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:54.638245 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:54.638917 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:54.638503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:54.638917 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:54.638547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:54.638917 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:54.638398 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:57:54.638917 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:54.638576 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:57:54.638917 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:54.638547 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:57:54.638917 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:54.638620 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls podName:ed424306-bd32-44c0-ac6b-354738a1eedd nodeName:}" failed. No retries permitted until 2026-07-06 23:57:58.638605469 +0000 UTC m=+44.611090122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls") pod "dns-default-7xhjt" (UID: "ed424306-bd32-44c0-ac6b-354738a1eedd") : secret "dns-default-metrics-tls" not found Jul 06 23:57:54.638917 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:54.638634 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm podName:05598b83-7ee6-42f2-977c-a41b3738a4b7 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:58.638627446 +0000 UTC m=+44.611112086 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tr5kc" (UID: "05598b83-7ee6-42f2-977c-a41b3738a4b7") : configmap "kube-rbac-proxy" not found Jul 06 23:57:54.638917 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:54.638667 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls podName:05598b83-7ee6-42f2-977c-a41b3738a4b7 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:58.638644914 +0000 UTC m=+44.611129558 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tr5kc" (UID: "05598b83-7ee6-42f2-977c-a41b3738a4b7") : secret "insights-runtime-extractor-tls" not found Jul 06 23:57:54.785304 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:54.785267 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvddn" event={"ID":"354c35b5-4b7a-47c3-825e-ce0decc20f9a","Type":"ContainerStarted","Data":"608161b82829ff8a6e1d5570e35110b13923a4e50ad790b144e6a3627163cecf"} Jul 06 23:57:54.786694 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:54.786667 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dprmh" event={"ID":"9d914f67-2aac-4cca-91ea-77dd7f0eca61","Type":"ContainerStarted","Data":"b353d78ffacc6ee5a8c740b223c1b86f19199276f300b2f69bcba95e1b23b9df"} Jul 06 23:57:54.788358 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:54.788329 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k" event={"ID":"fe3dbf5d-804e-4447-8bec-408cfe773dbf","Type":"ContainerStarted","Data":"caa32fc6337bdfbed858cad0f0391cb32e32e08096d65f6d1e2ecd4bd464761e"} Jul 06 23:57:54.788444 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:54.788375 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k" event={"ID":"fe3dbf5d-804e-4447-8bec-408cfe773dbf","Type":"ContainerStarted","Data":"2c4188712b3755b19a0af6bb7ce197a2a9a860e88981de187f4471efe5b17d12"} Jul 06 23:57:54.808347 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:54.808297 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bvddn" podStartSLOduration=11.026315602 podStartE2EDuration="40.808280832s" totalImagesPullingTime="29.78196523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:54.806585831 +0000 UTC m=+40.779070514" watchObservedRunningTime="2026-07-06 23:57:54.808280832 +0000 UTC m=+40.780765495" Jul 06 23:57:54.821854 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:54.821809 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dprmh" podStartSLOduration=2.488283914 podStartE2EDuration="4.821796515s" totalImagesPullingTime="2.333512601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:54.821457833 +0000 UTC m=+40.793942494" watchObservedRunningTime="2026-07-06 23:57:54.821796515 +0000 UTC m=+40.794281176" Jul 06 23:57:54.844525 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:54.844476 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-bn52k" podStartSLOduration=2.13133656 podStartE2EDuration="3.844465723s" totalImagesPullingTime="1.713129163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:54.843738626 +0000 UTC m=+40.816223286" watchObservedRunningTime="2026-07-06 23:57:54.844465723 +0000 UTC m=+40.816950658" Jul 06 23:57:56.607371 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.607336 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-ng6cf"] Jul 06 23:57:56.610414 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.610398 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-ng6cf" Jul 06 23:57:56.613111 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.613087 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-klfb2\"" Jul 06 23:57:56.613111 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.613099 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 23:57:56.613966 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.613950 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 23:57:56.614032 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.613950 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 23:57:56.614032 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.614017 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 23:57:56.618173 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.618155 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-ng6cf"] Jul 06 23:57:56.756634 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.756588 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/52714c1a-26fc-4189-8fbf-f63c9803708d-signing-key\") pod \"service-ca-758f599c5-ng6cf\" (UID: \"52714c1a-26fc-4189-8fbf-f63c9803708d\") " pod="openshift-service-ca/service-ca-758f599c5-ng6cf" Jul 06 23:57:56.756839 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.756711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb85n\" (UniqueName: \"kubernetes.io/projected/52714c1a-26fc-4189-8fbf-f63c9803708d-kube-api-access-zb85n\") pod \"service-ca-758f599c5-ng6cf\" (UID: \"52714c1a-26fc-4189-8fbf-f63c9803708d\") " pod="openshift-service-ca/service-ca-758f599c5-ng6cf" Jul 06 23:57:56.756839 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.756781 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/52714c1a-26fc-4189-8fbf-f63c9803708d-signing-cabundle\") pod \"service-ca-758f599c5-ng6cf\" (UID: \"52714c1a-26fc-4189-8fbf-f63c9803708d\") " pod="openshift-service-ca/service-ca-758f599c5-ng6cf" Jul 06 23:57:56.857712 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.857625 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/52714c1a-26fc-4189-8fbf-f63c9803708d-signing-cabundle\") pod \"service-ca-758f599c5-ng6cf\" (UID: \"52714c1a-26fc-4189-8fbf-f63c9803708d\") " pod="openshift-service-ca/service-ca-758f599c5-ng6cf" Jul 06 23:57:56.857712 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.857675 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/52714c1a-26fc-4189-8fbf-f63c9803708d-signing-key\") pod \"service-ca-758f599c5-ng6cf\" (UID: \"52714c1a-26fc-4189-8fbf-f63c9803708d\") " pod="openshift-service-ca/service-ca-758f599c5-ng6cf" Jul 06 23:57:56.857889 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.857752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb85n\" (UniqueName: \"kubernetes.io/projected/52714c1a-26fc-4189-8fbf-f63c9803708d-kube-api-access-zb85n\") pod \"service-ca-758f599c5-ng6cf\" (UID: \"52714c1a-26fc-4189-8fbf-f63c9803708d\") " pod="openshift-service-ca/service-ca-758f599c5-ng6cf" Jul 06 23:57:56.858345 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.858324 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/52714c1a-26fc-4189-8fbf-f63c9803708d-signing-cabundle\") pod \"service-ca-758f599c5-ng6cf\" (UID: \"52714c1a-26fc-4189-8fbf-f63c9803708d\") " pod="openshift-service-ca/service-ca-758f599c5-ng6cf" Jul 06 23:57:56.861224 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.861208 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/52714c1a-26fc-4189-8fbf-f63c9803708d-signing-key\") pod \"service-ca-758f599c5-ng6cf\" (UID: \"52714c1a-26fc-4189-8fbf-f63c9803708d\") " pod="openshift-service-ca/service-ca-758f599c5-ng6cf" Jul 06 23:57:56.867268 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.867243 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb85n\" (UniqueName: \"kubernetes.io/projected/52714c1a-26fc-4189-8fbf-f63c9803708d-kube-api-access-zb85n\") pod \"service-ca-758f599c5-ng6cf\" (UID: \"52714c1a-26fc-4189-8fbf-f63c9803708d\") " pod="openshift-service-ca/service-ca-758f599c5-ng6cf" Jul 06 23:57:56.920436 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:56.920396 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-ng6cf" Jul 06 23:57:57.049893 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:57.049858 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-ng6cf"] Jul 06 23:57:57.053178 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:57:57.053150 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52714c1a_26fc_4189_8fbf_f63c9803708d.slice/crio-5aec61dac1c95c65822d1baa8dcb4a626746ff7c6535bda7e689da5c801d6bfc WatchSource:0}: Error finding container 5aec61dac1c95c65822d1baa8dcb4a626746ff7c6535bda7e689da5c801d6bfc: Status 404 returned error can't find the container with id 5aec61dac1c95c65822d1baa8dcb4a626746ff7c6535bda7e689da5c801d6bfc Jul 06 23:57:57.797724 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:57.797677 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-ng6cf" event={"ID":"52714c1a-26fc-4189-8fbf-f63c9803708d","Type":"ContainerStarted","Data":"5aec61dac1c95c65822d1baa8dcb4a626746ff7c6535bda7e689da5c801d6bfc"} Jul 06 23:57:58.673218 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:58.673174 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:58.673218 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:58.673221 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:57:58.673452 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:58.673332 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:57:58.673452 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:58.673404 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:57:58.673452 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:58.673422 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm podName:05598b83-7ee6-42f2-977c-a41b3738a4b7 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:06.673396836 +0000 UTC m=+52.645881490 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tr5kc" (UID: "05598b83-7ee6-42f2-977c-a41b3738a4b7") : configmap "kube-rbac-proxy" not found Jul 06 23:57:58.673452 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:58.673445 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls podName:ed424306-bd32-44c0-ac6b-354738a1eedd nodeName:}" failed. No retries permitted until 2026-07-06 23:58:06.673432004 +0000 UTC m=+52.645916647 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls") pod "dns-default-7xhjt" (UID: "ed424306-bd32-44c0-ac6b-354738a1eedd") : secret "dns-default-metrics-tls" not found Jul 06 23:57:58.673606 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:58.673468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:57:58.673606 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:58.673566 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:57:58.673677 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:57:58.673612 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls podName:05598b83-7ee6-42f2-977c-a41b3738a4b7 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:06.673599566 +0000 UTC m=+52.646084219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tr5kc" (UID: "05598b83-7ee6-42f2-977c-a41b3738a4b7") : secret "insights-runtime-extractor-tls" not found Jul 06 23:57:59.803959 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:59.803919 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-ng6cf" event={"ID":"52714c1a-26fc-4189-8fbf-f63c9803708d","Type":"ContainerStarted","Data":"9dc00d9bb3dbc23bb5cbd34c6adc109dfb45e325cd86847ca993681ba4598f09"} Jul 06 23:57:59.822961 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:57:59.822906 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-ng6cf" podStartSLOduration=1.24975037 podStartE2EDuration="3.822889044s" totalImagesPullingTime="2.573138674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:59.822096884 +0000 UTC m=+45.794581549" watchObservedRunningTime="2026-07-06 23:57:59.822889044 +0000 UTC m=+45.795373732" Jul 06 23:58:06.741501 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:06.741458 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:58:06.741501 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:06.741506 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:58:06.742010 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:06.741618 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:58:06.742010 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:58:06.741718 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:58:06.742010 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:58:06.741779 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls podName:05598b83-7ee6-42f2-977c-a41b3738a4b7 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:22.741763216 +0000 UTC m=+68.714247857 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tr5kc" (UID: "05598b83-7ee6-42f2-977c-a41b3738a4b7") : secret "insights-runtime-extractor-tls" not found Jul 06 23:58:06.742146 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:06.742058 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05598b83-7ee6-42f2-977c-a41b3738a4b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:58:06.744132 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:06.744111 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed424306-bd32-44c0-ac6b-354738a1eedd-metrics-tls\") pod \"dns-default-7xhjt\" (UID: \"ed424306-bd32-44c0-ac6b-354738a1eedd\") " pod="openshift-dns/dns-default-7xhjt" Jul 06 23:58:06.761382 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:06.761354 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xhjt" Jul 06 23:58:06.912986 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:06.912956 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xhjt"] Jul 06 23:58:06.917591 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:58:06.917562 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded424306_bd32_44c0_ac6b_354738a1eedd.slice/crio-967635e44d33049078e4d720c737bbcef99eaa7d9e3f23d767a99fcf5d1346be WatchSource:0}: Error finding container 967635e44d33049078e4d720c737bbcef99eaa7d9e3f23d767a99fcf5d1346be: Status 404 returned error can't find the container with id 967635e44d33049078e4d720c737bbcef99eaa7d9e3f23d767a99fcf5d1346be Jul 06 23:58:07.828129 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:07.828081 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xhjt" event={"ID":"ed424306-bd32-44c0-ac6b-354738a1eedd","Type":"ContainerStarted","Data":"967635e44d33049078e4d720c737bbcef99eaa7d9e3f23d767a99fcf5d1346be"} Jul 06 23:58:09.837286 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:09.837252 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xhjt" event={"ID":"ed424306-bd32-44c0-ac6b-354738a1eedd","Type":"ContainerStarted","Data":"351a2ff5d21b5c62c41852fda9696994f335b3cee7b875883d47faf11429f4de"} Jul 06 23:58:09.837774 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:09.837304 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xhjt" event={"ID":"ed424306-bd32-44c0-ac6b-354738a1eedd","Type":"ContainerStarted","Data":"e7b5b97ae183645a092b36ff289990d47a3eb4c58309222bd56fc37442428e9d"} Jul 06 23:58:09.837774 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:09.837374 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7xhjt" Jul 06 23:58:09.856552 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:09.856502 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7xhjt" podStartSLOduration=17.632438656 podStartE2EDuration="19.856485429s" totalImagesPullingTime="2.224046773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:09.855447511 +0000 UTC m=+55.827932173" watchObservedRunningTime="2026-07-06 23:58:09.856485429 +0000 UTC m=+55.828970146" Jul 06 23:58:15.758676 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.758645 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c8f8d46d6-hmqcr"] Jul 06 23:58:15.784211 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.784175 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c8f8d46d6-hmqcr"] Jul 06 23:58:15.784385 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.784283 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr8m4" Jul 06 23:58:15.784450 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.784423 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.789541 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.789357 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 23:58:15.789541 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.789410 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qtrrj\"" Jul 06 23:58:15.789541 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.789362 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 23:58:15.789541 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.789477 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 23:58:15.796407 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.796387 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 23:58:15.806514 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.806487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f42f2861-0d59-4ac9-a6c7-a955b86059af-registry-certificates\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.806619 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.806519 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f42f2861-0d59-4ac9-a6c7-a955b86059af-image-registry-private-configuration\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.806619 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.806600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f42f2861-0d59-4ac9-a6c7-a955b86059af-installation-pull-secrets\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.806689 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.806653 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f42f2861-0d59-4ac9-a6c7-a955b86059af-bound-sa-token\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.806722 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.806687 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f42f2861-0d59-4ac9-a6c7-a955b86059af-ca-trust-extracted\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.806767 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.806749 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f42f2861-0d59-4ac9-a6c7-a955b86059af-registry-tls\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.806802 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.806779 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f42f2861-0d59-4ac9-a6c7-a955b86059af-trusted-ca\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.806833 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.806810 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8hp8\" (UniqueName: \"kubernetes.io/projected/f42f2861-0d59-4ac9-a6c7-a955b86059af-kube-api-access-p8hp8\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.907664 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.907634 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f42f2861-0d59-4ac9-a6c7-a955b86059af-installation-pull-secrets\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.907664 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.907664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f42f2861-0d59-4ac9-a6c7-a955b86059af-bound-sa-token\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.907941 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.907684 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f42f2861-0d59-4ac9-a6c7-a955b86059af-ca-trust-extracted\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.907941 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.907703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f42f2861-0d59-4ac9-a6c7-a955b86059af-registry-tls\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.907941 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.907717 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f42f2861-0d59-4ac9-a6c7-a955b86059af-trusted-ca\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.907941 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.907739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8hp8\" (UniqueName: \"kubernetes.io/projected/f42f2861-0d59-4ac9-a6c7-a955b86059af-kube-api-access-p8hp8\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.907941 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.907821 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f42f2861-0d59-4ac9-a6c7-a955b86059af-registry-certificates\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.907941 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.907848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f42f2861-0d59-4ac9-a6c7-a955b86059af-image-registry-private-configuration\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.908369 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.908191 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f42f2861-0d59-4ac9-a6c7-a955b86059af-ca-trust-extracted\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.908891 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.908864 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f42f2861-0d59-4ac9-a6c7-a955b86059af-registry-certificates\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.908958 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.908942 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f42f2861-0d59-4ac9-a6c7-a955b86059af-trusted-ca\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.910337 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.910317 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f42f2861-0d59-4ac9-a6c7-a955b86059af-image-registry-private-configuration\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.910704 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.910686 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f42f2861-0d59-4ac9-a6c7-a955b86059af-installation-pull-secrets\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.910813 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.910793 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f42f2861-0d59-4ac9-a6c7-a955b86059af-registry-tls\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.918134 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.918103 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8hp8\" (UniqueName: \"kubernetes.io/projected/f42f2861-0d59-4ac9-a6c7-a955b86059af-kube-api-access-p8hp8\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:15.919357 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:15.919333 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f42f2861-0d59-4ac9-a6c7-a955b86059af-bound-sa-token\") pod \"image-registry-7c8f8d46d6-hmqcr\" (UID: \"f42f2861-0d59-4ac9-a6c7-a955b86059af\") " pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:16.094831 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:16.094737 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:16.255484 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:16.255452 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c8f8d46d6-hmqcr"] Jul 06 23:58:16.258652 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:58:16.258627 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42f2861_0d59_4ac9_a6c7_a955b86059af.slice/crio-7fc3518ebd09d9cf7244fd53d56b642793b75f6e8c8410549c31a998408bd4b1 WatchSource:0}: Error finding container 7fc3518ebd09d9cf7244fd53d56b642793b75f6e8c8410549c31a998408bd4b1: Status 404 returned error can't find the container with id 7fc3518ebd09d9cf7244fd53d56b642793b75f6e8c8410549c31a998408bd4b1 Jul 06 23:58:16.860354 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:16.860319 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" event={"ID":"f42f2861-0d59-4ac9-a6c7-a955b86059af","Type":"ContainerStarted","Data":"1f19fb9f5081b1aec80c527ec4c0832ade45dcbc44c8cb3aeae7f9d8ac0c5e72"} Jul 06 23:58:16.860784 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:16.860367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" event={"ID":"f42f2861-0d59-4ac9-a6c7-a955b86059af","Type":"ContainerStarted","Data":"7fc3518ebd09d9cf7244fd53d56b642793b75f6e8c8410549c31a998408bd4b1"} Jul 06 23:58:16.860784 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:16.860453 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:16.882818 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:16.882770 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" podStartSLOduration=1.882752951 podStartE2EDuration="1.882752951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:16.881635547 +0000 UTC m=+62.854120235" watchObservedRunningTime="2026-07-06 23:58:16.882752951 +0000 UTC m=+62.855237611" Jul 06 23:58:19.842507 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:19.842483 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7xhjt" Jul 06 23:58:20.241583 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.241541 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:58:20.241789 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.241606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:58:20.244023 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.243999 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:58:20.244170 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.244032 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:58:20.254421 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.254396 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7888a0e8-ee58-4d66-8720-70feb41fef79-original-pull-secret\") pod \"global-pull-secret-syncer-9vrmj\" (UID: \"7888a0e8-ee58-4d66-8720-70feb41fef79\") " pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:58:20.254532 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.254466 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2f756d2-cf8a-463e-ac13-049452ff8425-metrics-certs\") pod \"network-metrics-daemon-k8l5b\" (UID: \"d2f756d2-cf8a-463e-ac13-049452ff8425\") " pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:58:20.393870 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.393830 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vrmj" Jul 06 23:58:20.401433 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.401404 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wdx8b\"" Jul 06 23:58:20.408596 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.408573 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8l5b" Jul 06 23:58:20.444104 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.443968 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bqb\" (UniqueName: \"kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb\") pod \"network-check-target-8ph76\" (UID: \"9d2f0b67-1dff-4670-8056-4023d2228f8c\") " pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:58:20.447146 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.447120 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:58:20.457136 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.457036 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:58:20.467593 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.467512 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5bqb\" (UniqueName: \"kubernetes.io/projected/9d2f0b67-1dff-4670-8056-4023d2228f8c-kube-api-access-b5bqb\") pod \"network-check-target-8ph76\" (UID: \"9d2f0b67-1dff-4670-8056-4023d2228f8c\") " pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:58:20.526636 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.526604 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9vrmj"] Jul 06 23:58:20.529267 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:58:20.529240 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7888a0e8_ee58_4d66_8720_70feb41fef79.slice/crio-23c02563020113bb2506a110f39e5ed3ea6894f0f35c1db81f6fee011bb71fce WatchSource:0}: Error finding container 23c02563020113bb2506a110f39e5ed3ea6894f0f35c1db81f6fee011bb71fce: Status 404 returned error can't find the container with id 23c02563020113bb2506a110f39e5ed3ea6894f0f35c1db81f6fee011bb71fce Jul 06 23:58:20.549203 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.546793 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k8l5b"] Jul 06 23:58:20.550036 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:58:20.550006 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f756d2_cf8a_463e_ac13_049452ff8425.slice/crio-f8a965d6f6452a8a703faecc6403c746f97566a807070b00a52ccd9053a7f742 WatchSource:0}: Error finding container f8a965d6f6452a8a703faecc6403c746f97566a807070b00a52ccd9053a7f742: Status 404 returned error can't find the container with id f8a965d6f6452a8a703faecc6403c746f97566a807070b00a52ccd9053a7f742 Jul 06 23:58:20.691730 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.691700 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v6xkb\"" Jul 06 23:58:20.698813 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.698789 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:58:20.825777 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.825746 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8ph76"] Jul 06 23:58:20.829308 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:58:20.829282 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2f0b67_1dff_4670_8056_4023d2228f8c.slice/crio-853ba09c0a941cb4fcb06786b0030e0524ab35d2ff17c1aa816d6102a7782b7a WatchSource:0}: Error finding container 853ba09c0a941cb4fcb06786b0030e0524ab35d2ff17c1aa816d6102a7782b7a: Status 404 returned error can't find the container with id 853ba09c0a941cb4fcb06786b0030e0524ab35d2ff17c1aa816d6102a7782b7a Jul 06 23:58:20.871984 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.871931 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k8l5b" event={"ID":"d2f756d2-cf8a-463e-ac13-049452ff8425","Type":"ContainerStarted","Data":"f8a965d6f6452a8a703faecc6403c746f97566a807070b00a52ccd9053a7f742"} Jul 06 23:58:20.873185 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.873157 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9vrmj" event={"ID":"7888a0e8-ee58-4d66-8720-70feb41fef79","Type":"ContainerStarted","Data":"23c02563020113bb2506a110f39e5ed3ea6894f0f35c1db81f6fee011bb71fce"} Jul 06 23:58:20.874178 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:20.874153 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8ph76" event={"ID":"9d2f0b67-1dff-4670-8056-4023d2228f8c","Type":"ContainerStarted","Data":"853ba09c0a941cb4fcb06786b0030e0524ab35d2ff17c1aa816d6102a7782b7a"} Jul 06 23:58:21.881648 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:21.881523 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k8l5b" event={"ID":"d2f756d2-cf8a-463e-ac13-049452ff8425","Type":"ContainerStarted","Data":"cffa7e0c428788e9af6e46cfc061c2a157921bbf8e6057107320792fb863a274"} Jul 06 23:58:21.881648 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:21.881595 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k8l5b" event={"ID":"d2f756d2-cf8a-463e-ac13-049452ff8425","Type":"ContainerStarted","Data":"80f3e23aec06f8687178c18dcbb6c9a840193a58f682331a457dde5ecbb9992c"} Jul 06 23:58:21.897778 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:21.897718 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k8l5b" podStartSLOduration=66.937258422 podStartE2EDuration="1m7.897699541s" totalImagesPullingTime="960.441119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:21.896948075 +0000 UTC m=+67.869432751" watchObservedRunningTime="2026-07-06 23:58:21.897699541 +0000 UTC m=+67.870184240" Jul 06 23:58:22.761325 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:22.761280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:58:22.764309 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:22.764278 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05598b83-7ee6-42f2-977c-a41b3738a4b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tr5kc\" (UID: \"05598b83-7ee6-42f2-977c-a41b3738a4b7\") " pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:58:23.003294 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:23.003264 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9lqq6\"" Jul 06 23:58:23.011452 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:23.011382 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tr5kc" Jul 06 23:58:23.158771 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:23.158739 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tr5kc"] Jul 06 23:58:24.084561 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:58:24.084513 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05598b83_7ee6_42f2_977c_a41b3738a4b7.slice/crio-1e06aa512fb7aed82879edff0087bbf59bbc6294d386978735f56a57d954ecdd WatchSource:0}: Error finding container 1e06aa512fb7aed82879edff0087bbf59bbc6294d386978735f56a57d954ecdd: Status 404 returned error can't find the container with id 1e06aa512fb7aed82879edff0087bbf59bbc6294d386978735f56a57d954ecdd Jul 06 23:58:24.896421 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:24.896377 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tr5kc" event={"ID":"05598b83-7ee6-42f2-977c-a41b3738a4b7","Type":"ContainerStarted","Data":"1e06aa512fb7aed82879edff0087bbf59bbc6294d386978735f56a57d954ecdd"} Jul 06 23:58:25.315683 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.315599 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c4d6978fd-gbpml"] Jul 06 23:58:25.339748 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.339713 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c4d6978fd-gbpml"] Jul 06 23:58:25.339939 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.339842 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.342545 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.342516 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 23:58:25.342683 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.342571 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 23:58:25.342683 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.342571 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 23:58:25.342925 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.342903 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 23:58:25.343815 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.343794 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-whz66\"" Jul 06 23:58:25.343913 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.343867 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 23:58:25.344125 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.343999 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 23:58:25.344125 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.343801 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 23:58:25.382322 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.382278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-oauth-serving-cert\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.382499 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.382352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-config\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.382499 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.382389 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-service-ca\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.382499 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.382439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-serving-cert\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.382499 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.382470 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkqz6\" (UniqueName: \"kubernetes.io/projected/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-kube-api-access-qkqz6\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.382499 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.382493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-oauth-config\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.483909 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.483871 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-config\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.484107 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.483915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-service-ca\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.484107 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.483937 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-serving-cert\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.484107 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.483954 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkqz6\" (UniqueName: \"kubernetes.io/projected/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-kube-api-access-qkqz6\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.484107 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.483970 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-oauth-config\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.484107 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.484008 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-oauth-serving-cert\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.484782 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.484753 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-config\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.484889 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.484763 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-oauth-serving-cert\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.484889 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.484818 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-service-ca\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.486411 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.486386 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-oauth-config\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.486625 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.486605 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-serving-cert\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.492084 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.492051 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkqz6\" (UniqueName: \"kubernetes.io/projected/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-kube-api-access-qkqz6\") pod \"console-5c4d6978fd-gbpml\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:25.652290 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:25.652240 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:26.055042 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:26.055007 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c4d6978fd-gbpml"] Jul 06 23:58:26.059166 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:58:26.059135 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee73fb1_4cc8_4fe6_8224_23136e18e6d7.slice/crio-ee8536fc27cfd25dfd94320b6eec734773a2b6e0922530896e5c6fa944bd9791 WatchSource:0}: Error finding container ee8536fc27cfd25dfd94320b6eec734773a2b6e0922530896e5c6fa944bd9791: Status 404 returned error can't find the container with id ee8536fc27cfd25dfd94320b6eec734773a2b6e0922530896e5c6fa944bd9791 Jul 06 23:58:26.907359 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:26.907314 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9vrmj" event={"ID":"7888a0e8-ee58-4d66-8720-70feb41fef79","Type":"ContainerStarted","Data":"fda239cca022b6270c6c1be426755c205b2f963be574e2eb67d0278a5d479478"} Jul 06 23:58:26.909832 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:26.909677 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4d6978fd-gbpml" event={"ID":"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7","Type":"ContainerStarted","Data":"ee8536fc27cfd25dfd94320b6eec734773a2b6e0922530896e5c6fa944bd9791"} Jul 06 23:58:26.911502 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:26.911472 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8ph76" event={"ID":"9d2f0b67-1dff-4670-8056-4023d2228f8c","Type":"ContainerStarted","Data":"2718a0741e127abf83dfe189e9306d6878030194a810bff5ddbfbbfddeff5c26"} Jul 06 23:58:26.911608 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:26.911547 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:58:26.922611 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:26.922554 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9vrmj" podStartSLOduration=67.558259448 podStartE2EDuration="1m12.922535015s" totalImagesPullingTime="5.364275567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:26.922450601 +0000 UTC m=+72.894935266" watchObservedRunningTime="2026-07-06 23:58:26.922535015 +0000 UTC m=+72.895019673" Jul 06 23:58:26.941298 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:26.941238 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8ph76" podStartSLOduration=67.88140682 podStartE2EDuration="1m12.941213585s" totalImagesPullingTime="5.059806765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:26.940448665 +0000 UTC m=+72.912933340" watchObservedRunningTime="2026-07-06 23:58:26.941213585 +0000 UTC m=+72.913698248" Jul 06 23:58:30.927669 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:30.927625 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4d6978fd-gbpml" event={"ID":"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7","Type":"ContainerStarted","Data":"6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128"} Jul 06 23:58:30.929112 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:30.929083 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tr5kc" event={"ID":"05598b83-7ee6-42f2-977c-a41b3738a4b7","Type":"ContainerStarted","Data":"cb0a20b70e4ab45c1fbf946e09e7b55b0383222dd0667fa680afdc380d7d9c59"} Jul 06 23:58:30.948466 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:30.948412 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c4d6978fd-gbpml" podStartSLOduration=1.468679229 podStartE2EDuration="5.94839485s" totalImagesPullingTime="4.479715621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:30.94818978 +0000 UTC m=+76.920674441" watchObservedRunningTime="2026-07-06 23:58:30.94839485 +0000 UTC m=+76.920879511" Jul 06 23:58:31.935817 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:31.935773 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tr5kc" event={"ID":"05598b83-7ee6-42f2-977c-a41b3738a4b7","Type":"ContainerStarted","Data":"399d155abf33441f8d0cb505fdec52511819a68846360b63c9af81b9094ea487"} Jul 06 23:58:33.949142 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:33.949109 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tr5kc" event={"ID":"05598b83-7ee6-42f2-977c-a41b3738a4b7","Type":"ContainerStarted","Data":"f968ee79a66472c155d881df45cc0c7da66dd86eac942fc0e95e4f8a8c02fe9c"} Jul 06 23:58:33.969916 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:33.969809 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tr5kc" podStartSLOduration=34.458493209 podStartE2EDuration="43.969793419s" totalImagesPullingTime="9.51130021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:33.9681078 +0000 UTC m=+79.940592462" watchObservedRunningTime="2026-07-06 23:58:33.969793419 +0000 UTC m=+79.942278081" Jul 06 23:58:35.653234 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:35.653191 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:35.653632 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:35.653270 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:35.657918 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:35.657893 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:35.959510 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:35.959371 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:58:37.867746 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:37.867714 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c8f8d46d6-hmqcr" Jul 06 23:58:38.589937 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.589902 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6"] Jul 06 23:58:38.594735 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.594714 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.597963 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.597937 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-x2ghg"] Jul 06 23:58:38.598096 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.598052 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 23:58:38.598589 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.598560 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 23:58:38.598711 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.598695 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 23:58:38.598782 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.598749 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 23:58:38.599607 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.599584 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jcpr9\"" Jul 06 23:58:38.599717 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.599679 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 23:58:38.599826 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.599788 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 23:58:38.601332 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.601314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.603670 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.603654 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 23:58:38.604338 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.604303 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 23:58:38.604461 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.604442 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-r62mn\"" Jul 06 23:58:38.604603 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.604584 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 23:58:38.604724 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.604701 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6"] Jul 06 23:58:38.692230 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.692230 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-textfile\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.692466 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692300 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.692466 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692336 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-tls\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.692466 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.692466 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692409 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f5fa57d-9dae-461c-a129-dda4f8f24e95-sys\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.692466 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692435 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f5fa57d-9dae-461c-a129-dda4f8f24e95-metrics-client-ca\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.692660 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692525 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-wtmp\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.692660 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692565 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-accelerators-collector-config\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.692660 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692605 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.692660 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f5fa57d-9dae-461c-a129-dda4f8f24e95-root\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.692789 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.692789 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692698 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lvkl\" (UniqueName: \"kubernetes.io/projected/6f5fa57d-9dae-461c-a129-dda4f8f24e95-kube-api-access-7lvkl\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.692789 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692757 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.692789 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.692781 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj6km\" (UniqueName: \"kubernetes.io/projected/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-api-access-fj6km\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.793481 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793442 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.793481 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793482 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f5fa57d-9dae-461c-a129-dda4f8f24e95-root\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.793718 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.793718 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvkl\" (UniqueName: \"kubernetes.io/projected/6f5fa57d-9dae-461c-a129-dda4f8f24e95-kube-api-access-7lvkl\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.793718 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793564 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.793718 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793580 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f5fa57d-9dae-461c-a129-dda4f8f24e95-root\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.793718 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793590 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj6km\" (UniqueName: \"kubernetes.io/projected/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-api-access-fj6km\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.793718 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:58:38.793629 2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 23:58:38.793718 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.793718 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:58:38.793706 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-tls podName:93151fb7-0fa3-4ce4-ba37-78cd690e6c6b nodeName:}" failed. No retries permitted until 2026-07-06 23:58:39.29368542 +0000 UTC m=+85.266170063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-dvlk6" (UID: "93151fb7-0fa3-4ce4-ba37-78cd690e6c6b") : secret "kube-state-metrics-tls" not found Jul 06 23:58:38.794139 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793725 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-textfile\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.794139 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.794139 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-tls\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.794139 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793827 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.794139 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793853 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f5fa57d-9dae-461c-a129-dda4f8f24e95-sys\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.794139 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f5fa57d-9dae-461c-a129-dda4f8f24e95-metrics-client-ca\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.794139 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.793954 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-wtmp\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.794139 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.794116 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-wtmp\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.794503 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:58:38.794171 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 23:58:38.794503 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:58:38.794231 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-tls podName:6f5fa57d-9dae-461c-a129-dda4f8f24e95 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:39.294214765 +0000 UTC m=+85.266699423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-tls") pod "node-exporter-x2ghg" (UID: "6f5fa57d-9dae-461c-a129-dda4f8f24e95") : secret "node-exporter-tls" not found Jul 06 23:58:38.794503 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.794253 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.794503 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.794291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-accelerators-collector-config\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.794720 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.794698 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-textfile\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.794797 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.794778 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f5fa57d-9dae-461c-a129-dda4f8f24e95-sys\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.794871 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.794850 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-accelerators-collector-config\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.795209 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.795187 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.795286 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.795267 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f5fa57d-9dae-461c-a129-dda4f8f24e95-metrics-client-ca\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.795459 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.795434 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.796618 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.796569 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:38.797359 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.797337 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.813472 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.813406 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lvkl\" (UniqueName: \"kubernetes.io/projected/6f5fa57d-9dae-461c-a129-dda4f8f24e95-kube-api-access-7lvkl\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:38.814546 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:38.814492 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj6km\" (UniqueName: \"kubernetes.io/projected/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-api-access-fj6km\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:39.299129 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:39.299094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:39.299522 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:39.299146 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-tls\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:39.301676 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:39.301636 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f5fa57d-9dae-461c-a129-dda4f8f24e95-node-exporter-tls\") pod \"node-exporter-x2ghg\" (UID: \"6f5fa57d-9dae-461c-a129-dda4f8f24e95\") " pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:39.301812 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:39.301677 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93151fb7-0fa3-4ce4-ba37-78cd690e6c6b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-dvlk6\" (UID: \"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:39.507538 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:39.507493 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" Jul 06 23:58:39.515166 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:39.515128 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x2ghg" Jul 06 23:58:39.527519 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:58:39.527483 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5fa57d_9dae_461c_a129_dda4f8f24e95.slice/crio-f657b6afa5e596b27a7dceedfedaba7ce0140811977539dbfb708c4408c8251b WatchSource:0}: Error finding container f657b6afa5e596b27a7dceedfedaba7ce0140811977539dbfb708c4408c8251b: Status 404 returned error can't find the container with id f657b6afa5e596b27a7dceedfedaba7ce0140811977539dbfb708c4408c8251b Jul 06 23:58:39.683848 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:39.683808 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6"] Jul 06 23:58:39.686726 ip-10-0-130-52 kubenswrapper[2562]: W0706 23:58:39.686695 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93151fb7_0fa3_4ce4_ba37_78cd690e6c6b.slice/crio-978201aec9251d24bbbacb3c46ef9b3e238cfd184fc3e6f7cc01886ace55f44d WatchSource:0}: Error finding container 978201aec9251d24bbbacb3c46ef9b3e238cfd184fc3e6f7cc01886ace55f44d: Status 404 returned error can't find the container with id 978201aec9251d24bbbacb3c46ef9b3e238cfd184fc3e6f7cc01886ace55f44d Jul 06 23:58:39.970430 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:39.970398 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x2ghg" event={"ID":"6f5fa57d-9dae-461c-a129-dda4f8f24e95","Type":"ContainerStarted","Data":"f657b6afa5e596b27a7dceedfedaba7ce0140811977539dbfb708c4408c8251b"} Jul 06 23:58:39.971606 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:39.971574 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" event={"ID":"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b","Type":"ContainerStarted","Data":"978201aec9251d24bbbacb3c46ef9b3e238cfd184fc3e6f7cc01886ace55f44d"} Jul 06 23:58:40.981186 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:40.981094 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x2ghg" event={"ID":"6f5fa57d-9dae-461c-a129-dda4f8f24e95","Type":"ContainerStarted","Data":"b2318c4f4e6b16e3adc5b9b1dc74f5fa906c046e311b8778ca369045b42a426c"} Jul 06 23:58:41.986868 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:41.986829 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" event={"ID":"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b","Type":"ContainerStarted","Data":"c3ff1cfa528503e29acfa4ae45634dea30bb3d6cfcc788ea89b08a099650a100"} Jul 06 23:58:41.987422 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:41.986903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" event={"ID":"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b","Type":"ContainerStarted","Data":"5fc88d87cf0c59868fe071a19500ceb8fcf159e8a7dcf92ad5ba03ecb03b2f23"} Jul 06 23:58:41.987422 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:41.986938 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" event={"ID":"93151fb7-0fa3-4ce4-ba37-78cd690e6c6b","Type":"ContainerStarted","Data":"c9592a36000a50d959327cd23c9db9618ba0f8ec2d491d48dd3af20248b57833"} Jul 06 23:58:41.988439 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:41.988418 2562 generic.go:356] "Generic (PLEG): container finished" podID="6f5fa57d-9dae-461c-a129-dda4f8f24e95" containerID="b2318c4f4e6b16e3adc5b9b1dc74f5fa906c046e311b8778ca369045b42a426c" exitCode=0 Jul 06 23:58:41.988510 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:41.988450 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x2ghg" event={"ID":"6f5fa57d-9dae-461c-a129-dda4f8f24e95","Type":"ContainerDied","Data":"b2318c4f4e6b16e3adc5b9b1dc74f5fa906c046e311b8778ca369045b42a426c"} Jul 06 23:58:42.005414 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:42.005370 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-dvlk6" podStartSLOduration=2.536315288 podStartE2EDuration="4.005353865s" totalImagesPullingTime="1.469038577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:42.004530165 +0000 UTC m=+87.977014846" watchObservedRunningTime="2026-07-06 23:58:42.005353865 +0000 UTC m=+87.977838527" Jul 06 23:58:42.993722 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:42.993674 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x2ghg" event={"ID":"6f5fa57d-9dae-461c-a129-dda4f8f24e95","Type":"ContainerStarted","Data":"b83089368f8cdf6fbc9b299a7ea3256fb7c8016194778dae8779a5d9ffa54ddb"} Jul 06 23:58:42.994133 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:42.993743 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x2ghg" event={"ID":"6f5fa57d-9dae-461c-a129-dda4f8f24e95","Type":"ContainerStarted","Data":"aa5740c6acc8ceaa1df346beaeef897e7b14ccab87816ff10066ec551fac5faf"} Jul 06 23:58:43.014408 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:43.014353 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-x2ghg" podStartSLOduration=3.814803214 podStartE2EDuration="5.014337107s" totalImagesPullingTime="1.126719628s" totalInitContainerRuntime="72.814265ms" isStatefulPod=true podCreationTimestamp="2026-07-06 23:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:43.01250394 +0000 UTC m=+88.984988626" watchObservedRunningTime="2026-07-06 23:58:43.014337107 +0000 UTC m=+88.986821767" Jul 06 23:58:43.203046 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:43.203008 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c4d6978fd-gbpml"] Jul 06 23:58:57.918447 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:58:57.918417 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8ph76" Jul 06 23:59:08.224180 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.224110 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5c4d6978fd-gbpml" podUID="1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" containerName="console" containerID="cri-o://6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128" gracePeriod=15 Jul 06 23:59:08.472690 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.472663 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:59:08.531594 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.531486 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-config\") pod \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " Jul 06 23:59:08.531594 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.531531 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-oauth-serving-cert\") pod \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " Jul 06 23:59:08.531594 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.531550 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-service-ca\") pod \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " Jul 06 23:59:08.531594 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.531580 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-serving-cert\") pod \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " Jul 06 23:59:08.531930 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.531609 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-oauth-config\") pod \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " Jul 06 23:59:08.531930 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.531756 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkqz6\" (UniqueName: \"kubernetes.io/projected/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-kube-api-access-qkqz6\") pod \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\" (UID: \"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7\") " Jul 06 23:59:08.532033 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.532011 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-service-ca" (OuterVolumeSpecName: "service-ca") pod "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" (UID: "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:59:08.532033 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.532020 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" (UID: "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:59:08.532147 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.532044 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-config" (OuterVolumeSpecName: "console-config") pod "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" (UID: "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:59:08.534053 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.534024 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" (UID: "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:59:08.534053 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.534038 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-kube-api-access-qkqz6" (OuterVolumeSpecName: "kube-api-access-qkqz6") pod "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" (UID: "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7"). InnerVolumeSpecName "kube-api-access-qkqz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:59:08.534253 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.534094 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" (UID: "1ee73fb1-4cc8-4fe6-8224-23136e18e6d7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:59:08.633167 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.633135 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-serving-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 06 23:59:08.633167 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.633166 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-oauth-config\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 06 23:59:08.633385 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.633182 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkqz6\" (UniqueName: \"kubernetes.io/projected/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-kube-api-access-qkqz6\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 06 23:59:08.633385 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.633204 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-console-config\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 06 23:59:08.633385 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.633216 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-oauth-serving-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 06 23:59:08.633385 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:08.633228 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 06 23:59:09.085783 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:09.085748 2562 generic.go:356] "Generic (PLEG): container finished" podID="1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" containerID="6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128" exitCode=0 Jul 06 23:59:09.085951 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:09.085794 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4d6978fd-gbpml" Jul 06 23:59:09.085951 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:09.085831 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4d6978fd-gbpml" event={"ID":"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7","Type":"ContainerDied","Data":"6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128"} Jul 06 23:59:09.085951 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:09.085880 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4d6978fd-gbpml" event={"ID":"1ee73fb1-4cc8-4fe6-8224-23136e18e6d7","Type":"ContainerDied","Data":"ee8536fc27cfd25dfd94320b6eec734773a2b6e0922530896e5c6fa944bd9791"} Jul 06 23:59:09.085951 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:09.085899 2562 scope.go:117] "RemoveContainer" containerID="6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128" Jul 06 23:59:09.098430 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:09.098412 2562 scope.go:117] "RemoveContainer" containerID="6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128" Jul 06 23:59:09.098732 ip-10-0-130-52 kubenswrapper[2562]: E0706 23:59:09.098709 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128\": container with ID starting with 6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128 not found: ID does not exist" containerID="6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128" Jul 06 23:59:09.098810 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:09.098740 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128"} err="failed to get container status \"6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128\": rpc error: code = NotFound desc = could not find container \"6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128\": container with ID starting with 6fd8cdd19ce7cd27b8b3629432b6886a2bc5582327fc5cd08ec2f77851a06128 not found: ID does not exist" Jul 06 23:59:09.103861 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:09.103836 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c4d6978fd-gbpml"] Jul 06 23:59:09.107238 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:09.107217 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c4d6978fd-gbpml"] Jul 06 23:59:10.582342 ip-10-0-130-52 kubenswrapper[2562]: I0706 23:59:10.582309 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" path="/var/lib/kubelet/pods/1ee73fb1-4cc8-4fe6-8224-23136e18e6d7/volumes" Jul 07 00:00:00.158146 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.158113 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29723040-6dzvd"] Jul 07 00:00:00.158611 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.158371 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" containerName="console" Jul 07 00:00:00.158611 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.158384 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" containerName="console" Jul 07 00:00:00.158611 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.158424 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ee73fb1-4cc8-4fe6-8224-23136e18e6d7" containerName="console" Jul 07 00:00:00.161261 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.161246 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-6dzvd" Jul 07 00:00:00.165220 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.165199 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\"" Jul 07 00:00:00.165367 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.165259 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-jsshp\"" Jul 07 00:00:00.185573 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.185549 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29723040-6dzvd"] Jul 07 00:00:00.302719 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.302680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57e0b7f3-6007-402f-8624-d7edad486846-serviceca\") pod \"image-pruner-29723040-6dzvd\" (UID: \"57e0b7f3-6007-402f-8624-d7edad486846\") " pod="openshift-image-registry/image-pruner-29723040-6dzvd" Jul 07 00:00:00.302882 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.302742 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwp4j\" (UniqueName: \"kubernetes.io/projected/57e0b7f3-6007-402f-8624-d7edad486846-kube-api-access-lwp4j\") pod \"image-pruner-29723040-6dzvd\" (UID: \"57e0b7f3-6007-402f-8624-d7edad486846\") " pod="openshift-image-registry/image-pruner-29723040-6dzvd" Jul 07 00:00:00.403894 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.403853 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57e0b7f3-6007-402f-8624-d7edad486846-serviceca\") pod \"image-pruner-29723040-6dzvd\" (UID: \"57e0b7f3-6007-402f-8624-d7edad486846\") " pod="openshift-image-registry/image-pruner-29723040-6dzvd" Jul 07 00:00:00.404016 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.403892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwp4j\" (UniqueName: \"kubernetes.io/projected/57e0b7f3-6007-402f-8624-d7edad486846-kube-api-access-lwp4j\") pod \"image-pruner-29723040-6dzvd\" (UID: \"57e0b7f3-6007-402f-8624-d7edad486846\") " pod="openshift-image-registry/image-pruner-29723040-6dzvd" Jul 07 00:00:00.404531 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.404488 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57e0b7f3-6007-402f-8624-d7edad486846-serviceca\") pod \"image-pruner-29723040-6dzvd\" (UID: \"57e0b7f3-6007-402f-8624-d7edad486846\") " pod="openshift-image-registry/image-pruner-29723040-6dzvd" Jul 07 00:00:00.417918 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.417857 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwp4j\" (UniqueName: \"kubernetes.io/projected/57e0b7f3-6007-402f-8624-d7edad486846-kube-api-access-lwp4j\") pod \"image-pruner-29723040-6dzvd\" (UID: \"57e0b7f3-6007-402f-8624-d7edad486846\") " pod="openshift-image-registry/image-pruner-29723040-6dzvd" Jul 07 00:00:00.486459 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.486419 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-6dzvd" Jul 07 00:00:00.632949 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:00.632918 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29723040-6dzvd"] Jul 07 00:00:00.636499 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:00:00.636471 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e0b7f3_6007_402f_8624_d7edad486846.slice/crio-bf3fa454f976e498e421793ea4ae12bdedcd7916ac48bff76f8589b120deb0f6 WatchSource:0}: Error finding container bf3fa454f976e498e421793ea4ae12bdedcd7916ac48bff76f8589b120deb0f6: Status 404 returned error can't find the container with id bf3fa454f976e498e421793ea4ae12bdedcd7916ac48bff76f8589b120deb0f6 Jul 07 00:00:01.260552 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:01.260517 2562 generic.go:356] "Generic (PLEG): container finished" podID="57e0b7f3-6007-402f-8624-d7edad486846" containerID="8002782d289653f44f23f577acf711e95bd9ccf719bf029c3321c2a65ed1b0aa" exitCode=0 Jul 07 00:00:01.260924 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:01.260585 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29723040-6dzvd" event={"ID":"57e0b7f3-6007-402f-8624-d7edad486846","Type":"ContainerDied","Data":"8002782d289653f44f23f577acf711e95bd9ccf719bf029c3321c2a65ed1b0aa"} Jul 07 00:00:01.260924 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:01.260626 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29723040-6dzvd" event={"ID":"57e0b7f3-6007-402f-8624-d7edad486846","Type":"ContainerStarted","Data":"bf3fa454f976e498e421793ea4ae12bdedcd7916ac48bff76f8589b120deb0f6"} Jul 07 00:00:02.384222 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.384197 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-6dzvd" Jul 07 00:00:02.517828 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.517714 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwp4j\" (UniqueName: \"kubernetes.io/projected/57e0b7f3-6007-402f-8624-d7edad486846-kube-api-access-lwp4j\") pod \"57e0b7f3-6007-402f-8624-d7edad486846\" (UID: \"57e0b7f3-6007-402f-8624-d7edad486846\") " Jul 07 00:00:02.517828 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.517776 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57e0b7f3-6007-402f-8624-d7edad486846-serviceca\") pod \"57e0b7f3-6007-402f-8624-d7edad486846\" (UID: \"57e0b7f3-6007-402f-8624-d7edad486846\") " Jul 07 00:00:02.518176 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.518153 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57e0b7f3-6007-402f-8624-d7edad486846-serviceca" (OuterVolumeSpecName: "serviceca") pod "57e0b7f3-6007-402f-8624-d7edad486846" (UID: "57e0b7f3-6007-402f-8624-d7edad486846"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:00:02.519973 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.519947 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57e0b7f3-6007-402f-8624-d7edad486846-kube-api-access-lwp4j" (OuterVolumeSpecName: "kube-api-access-lwp4j") pod "57e0b7f3-6007-402f-8624-d7edad486846" (UID: "57e0b7f3-6007-402f-8624-d7edad486846"). InnerVolumeSpecName "kube-api-access-lwp4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:00:02.618342 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.618309 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwp4j\" (UniqueName: \"kubernetes.io/projected/57e0b7f3-6007-402f-8624-d7edad486846-kube-api-access-lwp4j\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:00:02.618342 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.618335 2562 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57e0b7f3-6007-402f-8624-d7edad486846-serviceca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:00:02.996334 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.996300 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-59bd85dd-64knq"] Jul 07 00:00:02.996550 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.996538 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57e0b7f3-6007-402f-8624-d7edad486846" containerName="image-pruner" Jul 07 00:00:02.996608 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.996552 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="57e0b7f3-6007-402f-8624-d7edad486846" containerName="image-pruner" Jul 07 00:00:02.996608 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.996596 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="57e0b7f3-6007-402f-8624-d7edad486846" containerName="image-pruner" Jul 07 00:00:02.999664 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:02.999647 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.002039 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.002011 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 00:00:03.002188 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.002132 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 00:00:03.002281 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.002262 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 00:00:03.002597 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.002580 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-c248c\"" Jul 07 00:00:03.002597 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.002587 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 00:00:03.002872 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.002857 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 00:00:03.007758 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.007741 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 00:00:03.011998 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.011978 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-59bd85dd-64knq"] Jul 07 00:00:03.121855 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.121820 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-federate-client-tls\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.121855 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.121863 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21a28c8f-7194-4947-9842-26b99034fd8a-serving-certs-ca-bundle\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.122095 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.121886 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.122095 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.121906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l674\" (UniqueName: \"kubernetes.io/projected/21a28c8f-7194-4947-9842-26b99034fd8a-kube-api-access-8l674\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.122095 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.121979 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21a28c8f-7194-4947-9842-26b99034fd8a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.122095 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.122014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21a28c8f-7194-4947-9842-26b99034fd8a-metrics-client-ca\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.122095 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.122051 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-secret-telemeter-client\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.122095 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.122092 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-telemeter-client-tls\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.222881 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.222840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-secret-telemeter-client\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.222881 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.222880 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-telemeter-client-tls\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.223134 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.222915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-federate-client-tls\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.223134 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.222945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21a28c8f-7194-4947-9842-26b99034fd8a-serving-certs-ca-bundle\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.223134 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.222972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.223134 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.222998 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l674\" (UniqueName: \"kubernetes.io/projected/21a28c8f-7194-4947-9842-26b99034fd8a-kube-api-access-8l674\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.223134 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.223055 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21a28c8f-7194-4947-9842-26b99034fd8a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.223351 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.223205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21a28c8f-7194-4947-9842-26b99034fd8a-metrics-client-ca\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.223812 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.223787 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21a28c8f-7194-4947-9842-26b99034fd8a-serving-certs-ca-bundle\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.223981 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.223958 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21a28c8f-7194-4947-9842-26b99034fd8a-metrics-client-ca\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.224223 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.224200 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21a28c8f-7194-4947-9842-26b99034fd8a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.225561 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.225536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-telemeter-client-tls\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.225670 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.225651 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.225711 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.225685 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-federate-client-tls\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.225745 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.225733 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/21a28c8f-7194-4947-9842-26b99034fd8a-secret-telemeter-client\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.231242 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.231222 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l674\" (UniqueName: \"kubernetes.io/projected/21a28c8f-7194-4947-9842-26b99034fd8a-kube-api-access-8l674\") pod \"telemeter-client-59bd85dd-64knq\" (UID: \"21a28c8f-7194-4947-9842-26b99034fd8a\") " pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.268048 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.267955 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29723040-6dzvd" event={"ID":"57e0b7f3-6007-402f-8624-d7edad486846","Type":"ContainerDied","Data":"bf3fa454f976e498e421793ea4ae12bdedcd7916ac48bff76f8589b120deb0f6"} Jul 07 00:00:03.268048 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.268003 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf3fa454f976e498e421793ea4ae12bdedcd7916ac48bff76f8589b120deb0f6" Jul 07 00:00:03.268048 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.267965 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-6dzvd" Jul 07 00:00:03.308890 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.308857 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" Jul 07 00:00:03.460010 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:03.459977 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-59bd85dd-64knq"] Jul 07 00:00:03.463305 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:00:03.463281 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21a28c8f_7194_4947_9842_26b99034fd8a.slice/crio-e5b382d9772dd8fe339c98428aa6b246e690ed34d62852d65bd5c052e35f9bc5 WatchSource:0}: Error finding container e5b382d9772dd8fe339c98428aa6b246e690ed34d62852d65bd5c052e35f9bc5: Status 404 returned error can't find the container with id e5b382d9772dd8fe339c98428aa6b246e690ed34d62852d65bd5c052e35f9bc5 Jul 07 00:00:04.273616 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:04.273577 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" event={"ID":"21a28c8f-7194-4947-9842-26b99034fd8a","Type":"ContainerStarted","Data":"e5b382d9772dd8fe339c98428aa6b246e690ed34d62852d65bd5c052e35f9bc5"} Jul 07 00:00:06.281224 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:06.281187 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" event={"ID":"21a28c8f-7194-4947-9842-26b99034fd8a","Type":"ContainerStarted","Data":"15788c06ac5c4da37d049fb29bd851a19477297e7a5b7d5e9047164eef008034"} Jul 07 00:00:07.287839 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:07.287801 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" event={"ID":"21a28c8f-7194-4947-9842-26b99034fd8a","Type":"ContainerStarted","Data":"398d9bba37c8befd3215c1f93224c6ab96188e1439c5f2ff0ec873eab1256fb2"} Jul 07 00:00:07.288237 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:07.287860 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" event={"ID":"21a28c8f-7194-4947-9842-26b99034fd8a","Type":"ContainerStarted","Data":"22bdd0671276ffd2f025cd3cfd7a3e0853cc1cae655b7aab54b69bbcccc67549"} Jul 07 00:00:07.319861 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:00:07.319753 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-59bd85dd-64knq" podStartSLOduration=1.804027166 podStartE2EDuration="5.319729233s" totalImagesPullingTime="3.515702067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:00:07.31711006 +0000 UTC m=+173.289594723" watchObservedRunningTime="2026-07-07 00:00:07.319729233 +0000 UTC m=+173.292213895" Jul 07 00:02:14.445431 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:14.445392 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 00:02:22.534640 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.534601 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm"] Jul 07 00:02:22.536753 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.536734 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm" Jul 07 00:02:22.539519 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.539501 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7blm7\"" Jul 07 00:02:22.540406 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.540376 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 00:02:22.540546 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.540496 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 00:02:22.540614 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.540562 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 00:02:22.540676 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.540654 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 07 00:02:22.550098 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.550054 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm"] Jul 07 00:02:22.614079 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.614028 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x"] Jul 07 00:02:22.616044 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.616020 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:22.618883 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.618851 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 00:02:22.626340 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.626315 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x"] Jul 07 00:02:22.639632 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.639606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d90be851-09af-4f22-b262-d0291af00e6e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6c879c7688-dhmsm\" (UID: \"d90be851-09af-4f22-b262-d0291af00e6e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm" Jul 07 00:02:22.639804 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.639646 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l76rz\" (UniqueName: \"kubernetes.io/projected/d90be851-09af-4f22-b262-d0291af00e6e-kube-api-access-l76rz\") pod \"managed-serviceaccount-addon-agent-6c879c7688-dhmsm\" (UID: \"d90be851-09af-4f22-b262-d0291af00e6e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm" Jul 07 00:02:22.740559 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.740525 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2a487d5-5828-4621-9cd9-cb3e322fd5d2-tmp\") pod \"klusterlet-addon-workmgr-77bb977dff-vgn6x\" (UID: \"f2a487d5-5828-4621-9cd9-cb3e322fd5d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:22.740753 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.740570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d90be851-09af-4f22-b262-d0291af00e6e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6c879c7688-dhmsm\" (UID: \"d90be851-09af-4f22-b262-d0291af00e6e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm" Jul 07 00:02:22.740753 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.740627 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-557v7\" (UniqueName: \"kubernetes.io/projected/f2a487d5-5828-4621-9cd9-cb3e322fd5d2-kube-api-access-557v7\") pod \"klusterlet-addon-workmgr-77bb977dff-vgn6x\" (UID: \"f2a487d5-5828-4621-9cd9-cb3e322fd5d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:22.740753 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.740660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l76rz\" (UniqueName: \"kubernetes.io/projected/d90be851-09af-4f22-b262-d0291af00e6e-kube-api-access-l76rz\") pod \"managed-serviceaccount-addon-agent-6c879c7688-dhmsm\" (UID: \"d90be851-09af-4f22-b262-d0291af00e6e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm" Jul 07 00:02:22.740753 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.740739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f2a487d5-5828-4621-9cd9-cb3e322fd5d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-77bb977dff-vgn6x\" (UID: \"f2a487d5-5828-4621-9cd9-cb3e322fd5d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:22.743144 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.743119 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d90be851-09af-4f22-b262-d0291af00e6e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6c879c7688-dhmsm\" (UID: \"d90be851-09af-4f22-b262-d0291af00e6e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm" Jul 07 00:02:22.750473 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.750440 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l76rz\" (UniqueName: \"kubernetes.io/projected/d90be851-09af-4f22-b262-d0291af00e6e-kube-api-access-l76rz\") pod \"managed-serviceaccount-addon-agent-6c879c7688-dhmsm\" (UID: \"d90be851-09af-4f22-b262-d0291af00e6e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm" Jul 07 00:02:22.841320 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.841220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f2a487d5-5828-4621-9cd9-cb3e322fd5d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-77bb977dff-vgn6x\" (UID: \"f2a487d5-5828-4621-9cd9-cb3e322fd5d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:22.841320 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.841294 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2a487d5-5828-4621-9cd9-cb3e322fd5d2-tmp\") pod \"klusterlet-addon-workmgr-77bb977dff-vgn6x\" (UID: \"f2a487d5-5828-4621-9cd9-cb3e322fd5d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:22.841552 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.841361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-557v7\" (UniqueName: \"kubernetes.io/projected/f2a487d5-5828-4621-9cd9-cb3e322fd5d2-kube-api-access-557v7\") pod \"klusterlet-addon-workmgr-77bb977dff-vgn6x\" (UID: \"f2a487d5-5828-4621-9cd9-cb3e322fd5d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:22.841745 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.841720 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2a487d5-5828-4621-9cd9-cb3e322fd5d2-tmp\") pod \"klusterlet-addon-workmgr-77bb977dff-vgn6x\" (UID: \"f2a487d5-5828-4621-9cd9-cb3e322fd5d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:22.843770 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.843748 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f2a487d5-5828-4621-9cd9-cb3e322fd5d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-77bb977dff-vgn6x\" (UID: \"f2a487d5-5828-4621-9cd9-cb3e322fd5d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:22.849618 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.849595 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-557v7\" (UniqueName: \"kubernetes.io/projected/f2a487d5-5828-4621-9cd9-cb3e322fd5d2-kube-api-access-557v7\") pod \"klusterlet-addon-workmgr-77bb977dff-vgn6x\" (UID: \"f2a487d5-5828-4621-9cd9-cb3e322fd5d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:22.860234 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.860211 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm" Jul 07 00:02:22.925459 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.925428 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:22.993153 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.993125 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm"] Jul 07 00:02:22.995572 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:02:22.995550 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd90be851_09af_4f22_b262_d0291af00e6e.slice/crio-9ab26f8ba0e4df7f16f2fb25136d8dd9f9791e3c5d184a50b0b009f20cda6f70 WatchSource:0}: Error finding container 9ab26f8ba0e4df7f16f2fb25136d8dd9f9791e3c5d184a50b0b009f20cda6f70: Status 404 returned error can't find the container with id 9ab26f8ba0e4df7f16f2fb25136d8dd9f9791e3c5d184a50b0b009f20cda6f70 Jul 07 00:02:22.997654 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:22.997633 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 00:02:23.065169 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:23.064993 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x"] Jul 07 00:02:23.067747 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:02:23.067715 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a487d5_5828_4621_9cd9_cb3e322fd5d2.slice/crio-1ea77160ed77fd431fa38582c4f7680143387610196580625b8004752b430214 WatchSource:0}: Error finding container 1ea77160ed77fd431fa38582c4f7680143387610196580625b8004752b430214: Status 404 returned error can't find the container with id 1ea77160ed77fd431fa38582c4f7680143387610196580625b8004752b430214 Jul 07 00:02:23.741081 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:23.741022 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm" event={"ID":"d90be851-09af-4f22-b262-d0291af00e6e","Type":"ContainerStarted","Data":"9ab26f8ba0e4df7f16f2fb25136d8dd9f9791e3c5d184a50b0b009f20cda6f70"} Jul 07 00:02:23.742919 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:23.742842 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" event={"ID":"f2a487d5-5828-4621-9cd9-cb3e322fd5d2","Type":"ContainerStarted","Data":"1ea77160ed77fd431fa38582c4f7680143387610196580625b8004752b430214"} Jul 07 00:02:25.759863 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.759818 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj"] Jul 07 00:02:25.763305 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.763280 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:25.765950 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.765925 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8qg9b\"" Jul 07 00:02:25.766924 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.766892 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 00:02:25.767060 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.766928 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 00:02:25.769145 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.769115 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:25.769579 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.769170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:25.769579 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.769242 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxjgm\" (UniqueName: \"kubernetes.io/projected/2f498787-0dbc-4195-a4e9-4eb85a019268-kube-api-access-gxjgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:25.771106 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.770934 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj"] Jul 07 00:02:25.870132 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.870089 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:25.870306 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.870155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:25.870306 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.870187 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxjgm\" (UniqueName: \"kubernetes.io/projected/2f498787-0dbc-4195-a4e9-4eb85a019268-kube-api-access-gxjgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:25.870542 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.870513 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:25.870674 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.870546 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:25.878894 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:25.878865 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxjgm\" (UniqueName: \"kubernetes.io/projected/2f498787-0dbc-4195-a4e9-4eb85a019268-kube-api-access-gxjgm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:26.076246 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:26.076155 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:27.173811 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:27.173651 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj"] Jul 07 00:02:27.177135 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:02:27.177108 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f498787_0dbc_4195_a4e9_4eb85a019268.slice/crio-b4c94c2c386b6b0752403c4c86220d0c2603a04da2db57eedfa95885bb0c629a WatchSource:0}: Error finding container b4c94c2c386b6b0752403c4c86220d0c2603a04da2db57eedfa95885bb0c629a: Status 404 returned error can't find the container with id b4c94c2c386b6b0752403c4c86220d0c2603a04da2db57eedfa95885bb0c629a Jul 07 00:02:27.769600 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:27.769555 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" event={"ID":"f2a487d5-5828-4621-9cd9-cb3e322fd5d2","Type":"ContainerStarted","Data":"7531ffb0bdc194e401280f57f356e245d2a37edaf14ac2ff0d3fdef33ae13036"} Jul 07 00:02:27.769821 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:27.769803 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:27.771484 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:27.771374 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" event={"ID":"2f498787-0dbc-4195-a4e9-4eb85a019268","Type":"ContainerStarted","Data":"b4c94c2c386b6b0752403c4c86220d0c2603a04da2db57eedfa95885bb0c629a"} Jul 07 00:02:27.771866 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:27.771831 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" Jul 07 00:02:27.773169 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:27.773143 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm" event={"ID":"d90be851-09af-4f22-b262-d0291af00e6e","Type":"ContainerStarted","Data":"6024567e6bb5399e6339481f928a09cfef38b7d0352eebf72d770e192b9fb4d2"} Jul 07 00:02:27.786329 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:27.786279 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77bb977dff-vgn6x" podStartSLOduration=1.807254864 podStartE2EDuration="5.786263396s" totalImagesPullingTime="3.979008532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:02:27.784994917 +0000 UTC m=+313.757479579" watchObservedRunningTime="2026-07-07 00:02:27.786263396 +0000 UTC m=+313.758748056" Jul 07 00:02:27.814407 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:27.814339 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c879c7688-dhmsm" podStartSLOduration=1.777277411 podStartE2EDuration="5.81431394s" totalImagesPullingTime="4.037036529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:02:27.814216736 +0000 UTC m=+313.786701399" watchObservedRunningTime="2026-07-07 00:02:27.81431394 +0000 UTC m=+313.786798602" Jul 07 00:02:34.801466 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:34.801431 2562 generic.go:356] "Generic (PLEG): container finished" podID="2f498787-0dbc-4195-a4e9-4eb85a019268" containerID="4979ffd2fe3197393324b544ab1ca3f527aaeb32642cc803e01f8c98ab8802d4" exitCode=0 Jul 07 00:02:34.801871 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:34.801501 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" event={"ID":"2f498787-0dbc-4195-a4e9-4eb85a019268","Type":"ContainerDied","Data":"4979ffd2fe3197393324b544ab1ca3f527aaeb32642cc803e01f8c98ab8802d4"} Jul 07 00:02:37.816451 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:37.816413 2562 generic.go:356] "Generic (PLEG): container finished" podID="2f498787-0dbc-4195-a4e9-4eb85a019268" containerID="22794fd120ddbd0de01873dd682f82e4148fc179e0ce853e502808394096d5f6" exitCode=0 Jul 07 00:02:37.816838 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:37.816492 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" event={"ID":"2f498787-0dbc-4195-a4e9-4eb85a019268","Type":"ContainerDied","Data":"22794fd120ddbd0de01873dd682f82e4148fc179e0ce853e502808394096d5f6"} Jul 07 00:02:46.853527 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:46.853491 2562 generic.go:356] "Generic (PLEG): container finished" podID="2f498787-0dbc-4195-a4e9-4eb85a019268" containerID="1ffd3f1137d5eac935f6d887166ab860e0efb34d077cfbf324072b7f9e7946d2" exitCode=0 Jul 07 00:02:46.853914 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:46.853532 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" event={"ID":"2f498787-0dbc-4195-a4e9-4eb85a019268","Type":"ContainerDied","Data":"1ffd3f1137d5eac935f6d887166ab860e0efb34d077cfbf324072b7f9e7946d2"} Jul 07 00:02:47.980638 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:47.980605 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:48.054269 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.054225 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxjgm\" (UniqueName: \"kubernetes.io/projected/2f498787-0dbc-4195-a4e9-4eb85a019268-kube-api-access-gxjgm\") pod \"2f498787-0dbc-4195-a4e9-4eb85a019268\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " Jul 07 00:02:48.054447 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.054292 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-bundle\") pod \"2f498787-0dbc-4195-a4e9-4eb85a019268\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " Jul 07 00:02:48.054447 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.054407 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-util\") pod \"2f498787-0dbc-4195-a4e9-4eb85a019268\" (UID: \"2f498787-0dbc-4195-a4e9-4eb85a019268\") " Jul 07 00:02:48.054962 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.054938 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-bundle" (OuterVolumeSpecName: "bundle") pod "2f498787-0dbc-4195-a4e9-4eb85a019268" (UID: "2f498787-0dbc-4195-a4e9-4eb85a019268"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:02:48.056533 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.056506 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f498787-0dbc-4195-a4e9-4eb85a019268-kube-api-access-gxjgm" (OuterVolumeSpecName: "kube-api-access-gxjgm") pod "2f498787-0dbc-4195-a4e9-4eb85a019268" (UID: "2f498787-0dbc-4195-a4e9-4eb85a019268"). InnerVolumeSpecName "kube-api-access-gxjgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:02:48.058731 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.058713 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-util" (OuterVolumeSpecName: "util") pod "2f498787-0dbc-4195-a4e9-4eb85a019268" (UID: "2f498787-0dbc-4195-a4e9-4eb85a019268"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:02:48.155791 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.155697 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:02:48.155791 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.155736 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f498787-0dbc-4195-a4e9-4eb85a019268-util\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:02:48.155791 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.155748 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxjgm\" (UniqueName: \"kubernetes.io/projected/2f498787-0dbc-4195-a4e9-4eb85a019268-kube-api-access-gxjgm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:02:48.861860 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.861833 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" Jul 07 00:02:48.862079 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.861828 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wpczj" event={"ID":"2f498787-0dbc-4195-a4e9-4eb85a019268","Type":"ContainerDied","Data":"b4c94c2c386b6b0752403c4c86220d0c2603a04da2db57eedfa95885bb0c629a"} Jul 07 00:02:48.862079 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:48.861947 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4c94c2c386b6b0752403c4c86220d0c2603a04da2db57eedfa95885bb0c629a" Jul 07 00:02:54.209193 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.209158 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r"] Jul 07 00:02:54.209588 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.209439 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f498787-0dbc-4195-a4e9-4eb85a019268" containerName="pull" Jul 07 00:02:54.209588 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.209450 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f498787-0dbc-4195-a4e9-4eb85a019268" containerName="pull" Jul 07 00:02:54.209588 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.209462 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f498787-0dbc-4195-a4e9-4eb85a019268" containerName="util" Jul 07 00:02:54.209588 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.209468 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f498787-0dbc-4195-a4e9-4eb85a019268" containerName="util" Jul 07 00:02:54.209588 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.209474 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f498787-0dbc-4195-a4e9-4eb85a019268" containerName="extract" Jul 07 00:02:54.209588 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.209479 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f498787-0dbc-4195-a4e9-4eb85a019268" containerName="extract" Jul 07 00:02:54.209588 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.209527 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f498787-0dbc-4195-a4e9-4eb85a019268" containerName="extract" Jul 07 00:02:54.270580 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.270540 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r"] Jul 07 00:02:54.270748 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.270670 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r" Jul 07 00:02:54.274386 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.274364 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 00:02:54.274555 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.274539 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-f7jzj\"" Jul 07 00:02:54.274634 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.274616 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 00:02:54.405436 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.405398 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff046d49-766e-42d2-a715-45c03a155a3f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vch9r\" (UID: \"ff046d49-766e-42d2-a715-45c03a155a3f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r" Jul 07 00:02:54.405631 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.405462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtg45\" (UniqueName: \"kubernetes.io/projected/ff046d49-766e-42d2-a715-45c03a155a3f-kube-api-access-vtg45\") pod \"cert-manager-operator-controller-manager-55c444bf86-vch9r\" (UID: \"ff046d49-766e-42d2-a715-45c03a155a3f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r" Jul 07 00:02:54.506366 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.506272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtg45\" (UniqueName: \"kubernetes.io/projected/ff046d49-766e-42d2-a715-45c03a155a3f-kube-api-access-vtg45\") pod \"cert-manager-operator-controller-manager-55c444bf86-vch9r\" (UID: \"ff046d49-766e-42d2-a715-45c03a155a3f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r" Jul 07 00:02:54.506366 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.506359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff046d49-766e-42d2-a715-45c03a155a3f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vch9r\" (UID: \"ff046d49-766e-42d2-a715-45c03a155a3f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r" Jul 07 00:02:54.506770 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.506751 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff046d49-766e-42d2-a715-45c03a155a3f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vch9r\" (UID: \"ff046d49-766e-42d2-a715-45c03a155a3f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r" Jul 07 00:02:54.515665 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.515640 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtg45\" (UniqueName: \"kubernetes.io/projected/ff046d49-766e-42d2-a715-45c03a155a3f-kube-api-access-vtg45\") pod \"cert-manager-operator-controller-manager-55c444bf86-vch9r\" (UID: \"ff046d49-766e-42d2-a715-45c03a155a3f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r" Jul 07 00:02:54.579566 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.579539 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r" Jul 07 00:02:54.773582 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.773504 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r"] Jul 07 00:02:54.774207 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:02:54.774174 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff046d49_766e_42d2_a715_45c03a155a3f.slice/crio-b2ad0909d16fe2b49ad4207b5e58cf5a8936c8da3064ffdd2713bb9d984b4655 WatchSource:0}: Error finding container b2ad0909d16fe2b49ad4207b5e58cf5a8936c8da3064ffdd2713bb9d984b4655: Status 404 returned error can't find the container with id b2ad0909d16fe2b49ad4207b5e58cf5a8936c8da3064ffdd2713bb9d984b4655 Jul 07 00:02:54.884590 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:54.884554 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r" event={"ID":"ff046d49-766e-42d2-a715-45c03a155a3f","Type":"ContainerStarted","Data":"b2ad0909d16fe2b49ad4207b5e58cf5a8936c8da3064ffdd2713bb9d984b4655"} Jul 07 00:02:57.897719 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:57.897682 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r" event={"ID":"ff046d49-766e-42d2-a715-45c03a155a3f","Type":"ContainerStarted","Data":"b56dea7fb1e30599a3bf40781261f10bf8cd1fa61e105372962502a5bd85a04b"} Jul 07 00:02:57.941478 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:57.941422 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vch9r" podStartSLOduration=1.859732852 podStartE2EDuration="3.941404509s" totalImagesPullingTime="2.081671657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:02:57.939183264 +0000 UTC m=+343.911667924" watchObservedRunningTime="2026-07-07 00:02:57.941404509 +0000 UTC m=+343.913889171" Jul 07 00:02:58.992880 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:58.992844 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2"] Jul 07 00:02:58.995046 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:58.995029 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:02:58.997560 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:58.997539 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 00:02:58.997795 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:58.997776 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 00:02:58.998918 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:58.998902 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8qg9b\"" Jul 07 00:02:59.025994 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.025957 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2"] Jul 07 00:02:59.149732 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.149697 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:02:59.149927 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.149747 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kddj\" (UniqueName: \"kubernetes.io/projected/8d32d187-e576-425f-89dd-8f3cf8381256-kube-api-access-2kddj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:02:59.149927 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.149801 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:02:59.251098 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.250984 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kddj\" (UniqueName: \"kubernetes.io/projected/8d32d187-e576-425f-89dd-8f3cf8381256-kube-api-access-2kddj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:02:59.251098 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.251027 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:02:59.251284 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.251116 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:02:59.251583 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.251560 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:02:59.251623 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.251571 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:02:59.277735 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.277708 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kddj\" (UniqueName: \"kubernetes.io/projected/8d32d187-e576-425f-89dd-8f3cf8381256-kube-api-access-2kddj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:02:59.304714 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.304663 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:02:59.454540 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.454508 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2"] Jul 07 00:02:59.457912 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:02:59.457884 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d32d187_e576_425f_89dd_8f3cf8381256.slice/crio-1455d4c470d6d06a8415f980b6cc3cfa0910069a875f462505b46f31dfc98d5e WatchSource:0}: Error finding container 1455d4c470d6d06a8415f980b6cc3cfa0910069a875f462505b46f31dfc98d5e: Status 404 returned error can't find the container with id 1455d4c470d6d06a8415f980b6cc3cfa0910069a875f462505b46f31dfc98d5e Jul 07 00:02:59.907165 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.907132 2562 generic.go:356] "Generic (PLEG): container finished" podID="8d32d187-e576-425f-89dd-8f3cf8381256" containerID="40dfd38a9707844efd69e2d2b2539ce80711cbd7f4b3d3710dc39632fc262530" exitCode=0 Jul 07 00:02:59.907351 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.907223 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" event={"ID":"8d32d187-e576-425f-89dd-8f3cf8381256","Type":"ContainerDied","Data":"40dfd38a9707844efd69e2d2b2539ce80711cbd7f4b3d3710dc39632fc262530"} Jul 07 00:02:59.907351 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:02:59.907281 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" event={"ID":"8d32d187-e576-425f-89dd-8f3cf8381256","Type":"ContainerStarted","Data":"1455d4c470d6d06a8415f980b6cc3cfa0910069a875f462505b46f31dfc98d5e"} Jul 07 00:03:05.195150 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.195118 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z5t5d"] Jul 07 00:03:05.198754 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.198733 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" Jul 07 00:03:05.202415 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.202389 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 00:03:05.203521 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.203497 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-72lvn\"" Jul 07 00:03:05.203639 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.203531 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 00:03:05.209963 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.209937 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z5t5d"] Jul 07 00:03:05.300232 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.300194 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b26b868-7e1e-48d7-9011-19dc076cbd62-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z5t5d\" (UID: \"0b26b868-7e1e-48d7-9011-19dc076cbd62\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" Jul 07 00:03:05.300418 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.300240 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vbs9\" (UniqueName: \"kubernetes.io/projected/0b26b868-7e1e-48d7-9011-19dc076cbd62-kube-api-access-7vbs9\") pod \"cert-manager-webhook-6486ffd997-z5t5d\" (UID: \"0b26b868-7e1e-48d7-9011-19dc076cbd62\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" Jul 07 00:03:05.401376 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.401343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b26b868-7e1e-48d7-9011-19dc076cbd62-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z5t5d\" (UID: \"0b26b868-7e1e-48d7-9011-19dc076cbd62\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" Jul 07 00:03:05.401590 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.401396 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vbs9\" (UniqueName: \"kubernetes.io/projected/0b26b868-7e1e-48d7-9011-19dc076cbd62-kube-api-access-7vbs9\") pod \"cert-manager-webhook-6486ffd997-z5t5d\" (UID: \"0b26b868-7e1e-48d7-9011-19dc076cbd62\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" Jul 07 00:03:05.415933 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.415908 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b26b868-7e1e-48d7-9011-19dc076cbd62-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z5t5d\" (UID: \"0b26b868-7e1e-48d7-9011-19dc076cbd62\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" Jul 07 00:03:05.417196 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.417176 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vbs9\" (UniqueName: \"kubernetes.io/projected/0b26b868-7e1e-48d7-9011-19dc076cbd62-kube-api-access-7vbs9\") pod \"cert-manager-webhook-6486ffd997-z5t5d\" (UID: \"0b26b868-7e1e-48d7-9011-19dc076cbd62\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" Jul 07 00:03:05.508426 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.508324 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" Jul 07 00:03:05.647642 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.647609 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z5t5d"] Jul 07 00:03:05.650771 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:03:05.650737 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b26b868_7e1e_48d7_9011_19dc076cbd62.slice/crio-eb70c83c3f52c45b7cd206e1b44c36f898a2840bcdd28ceaac85d6db5fd88d08 WatchSource:0}: Error finding container eb70c83c3f52c45b7cd206e1b44c36f898a2840bcdd28ceaac85d6db5fd88d08: Status 404 returned error can't find the container with id eb70c83c3f52c45b7cd206e1b44c36f898a2840bcdd28ceaac85d6db5fd88d08 Jul 07 00:03:05.932386 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:05.932353 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" event={"ID":"0b26b868-7e1e-48d7-9011-19dc076cbd62","Type":"ContainerStarted","Data":"eb70c83c3f52c45b7cd206e1b44c36f898a2840bcdd28ceaac85d6db5fd88d08"} Jul 07 00:03:09.953094 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:09.953026 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" event={"ID":"0b26b868-7e1e-48d7-9011-19dc076cbd62","Type":"ContainerStarted","Data":"d0c5dfcd923fc121eca594137ae470daea21f3feecc8627e17b85b669aa8d3a8"} Jul 07 00:03:09.953094 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:09.953098 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" Jul 07 00:03:09.954987 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:09.954958 2562 generic.go:356] "Generic (PLEG): container finished" podID="8d32d187-e576-425f-89dd-8f3cf8381256" containerID="e7c7e9da77c234060f9ac9a45f284a9d3414b9b26069395fdd7eab5176dce37b" exitCode=0 Jul 07 00:03:09.955114 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:09.955041 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" event={"ID":"8d32d187-e576-425f-89dd-8f3cf8381256","Type":"ContainerDied","Data":"e7c7e9da77c234060f9ac9a45f284a9d3414b9b26069395fdd7eab5176dce37b"} Jul 07 00:03:09.973620 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:09.973562 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" podStartSLOduration=1.7058935050000001 podStartE2EDuration="4.973535927s" totalImagesPullingTime="3.267642422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:09.97060728 +0000 UTC m=+355.943091941" watchObservedRunningTime="2026-07-07 00:03:09.973535927 +0000 UTC m=+355.946020589" Jul 07 00:03:10.961186 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:10.961151 2562 generic.go:356] "Generic (PLEG): container finished" podID="8d32d187-e576-425f-89dd-8f3cf8381256" containerID="f5033f6de92c53330f39d68f6e7b5cd2dc9d318e7e5e58a3f32239779d3a7b08" exitCode=0 Jul 07 00:03:10.961586 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:10.961233 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" event={"ID":"8d32d187-e576-425f-89dd-8f3cf8381256","Type":"ContainerDied","Data":"f5033f6de92c53330f39d68f6e7b5cd2dc9d318e7e5e58a3f32239779d3a7b08"} Jul 07 00:03:12.092145 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.092118 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:03:12.265061 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.264953 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kddj\" (UniqueName: \"kubernetes.io/projected/8d32d187-e576-425f-89dd-8f3cf8381256-kube-api-access-2kddj\") pod \"8d32d187-e576-425f-89dd-8f3cf8381256\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " Jul 07 00:03:12.265061 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.265000 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-util\") pod \"8d32d187-e576-425f-89dd-8f3cf8381256\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " Jul 07 00:03:12.265061 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.265054 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-bundle\") pod \"8d32d187-e576-425f-89dd-8f3cf8381256\" (UID: \"8d32d187-e576-425f-89dd-8f3cf8381256\") " Jul 07 00:03:12.265482 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.265451 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-bundle" (OuterVolumeSpecName: "bundle") pod "8d32d187-e576-425f-89dd-8f3cf8381256" (UID: "8d32d187-e576-425f-89dd-8f3cf8381256"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:03:12.267124 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.267098 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d32d187-e576-425f-89dd-8f3cf8381256-kube-api-access-2kddj" (OuterVolumeSpecName: "kube-api-access-2kddj") pod "8d32d187-e576-425f-89dd-8f3cf8381256" (UID: "8d32d187-e576-425f-89dd-8f3cf8381256"). InnerVolumeSpecName "kube-api-access-2kddj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:03:12.270414 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.270373 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-util" (OuterVolumeSpecName: "util") pod "8d32d187-e576-425f-89dd-8f3cf8381256" (UID: "8d32d187-e576-425f-89dd-8f3cf8381256"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:03:12.366677 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.366643 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:03:12.366677 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.366672 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kddj\" (UniqueName: \"kubernetes.io/projected/8d32d187-e576-425f-89dd-8f3cf8381256-kube-api-access-2kddj\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:03:12.366677 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.366682 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d32d187-e576-425f-89dd-8f3cf8381256-util\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:03:12.971137 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.971098 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" event={"ID":"8d32d187-e576-425f-89dd-8f3cf8381256","Type":"ContainerDied","Data":"1455d4c470d6d06a8415f980b6cc3cfa0910069a875f462505b46f31dfc98d5e"} Jul 07 00:03:12.971137 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.971138 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flrph2" Jul 07 00:03:12.971343 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:12.971154 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1455d4c470d6d06a8415f980b6cc3cfa0910069a875f462505b46f31dfc98d5e" Jul 07 00:03:15.963342 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:15.963309 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z5t5d" Jul 07 00:03:24.406631 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.406598 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk"] Jul 07 00:03:24.407096 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.406894 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d32d187-e576-425f-89dd-8f3cf8381256" containerName="extract" Jul 07 00:03:24.407096 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.406907 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d32d187-e576-425f-89dd-8f3cf8381256" containerName="extract" Jul 07 00:03:24.407096 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.406918 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d32d187-e576-425f-89dd-8f3cf8381256" containerName="pull" Jul 07 00:03:24.407096 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.406924 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d32d187-e576-425f-89dd-8f3cf8381256" containerName="pull" Jul 07 00:03:24.407096 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.406943 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d32d187-e576-425f-89dd-8f3cf8381256" containerName="util" Jul 07 00:03:24.407096 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.406949 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d32d187-e576-425f-89dd-8f3cf8381256" containerName="util" Jul 07 00:03:24.407096 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.406991 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d32d187-e576-425f-89dd-8f3cf8381256" containerName="extract" Jul 07 00:03:24.409236 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.409218 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:24.411819 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.411798 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 00:03:24.411944 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.411838 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8qg9b\"" Jul 07 00:03:24.412713 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.412695 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 00:03:24.428339 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.428308 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk"] Jul 07 00:03:24.561968 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.561930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:24.562159 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.561984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:24.562159 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.562030 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgsbl\" (UniqueName: \"kubernetes.io/projected/f8441d57-440f-496f-b9b9-40f7da4a0b90-kube-api-access-pgsbl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:24.662976 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.662873 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:24.662976 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.662945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:24.663263 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.662987 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgsbl\" (UniqueName: \"kubernetes.io/projected/f8441d57-440f-496f-b9b9-40f7da4a0b90-kube-api-access-pgsbl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:24.663336 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.663314 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:24.663382 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.663323 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:24.672828 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.672803 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgsbl\" (UniqueName: \"kubernetes.io/projected/f8441d57-440f-496f-b9b9-40f7da4a0b90-kube-api-access-pgsbl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:24.719465 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.719431 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:24.863976 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:24.863943 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk"] Jul 07 00:03:24.867051 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:03:24.867017 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8441d57_440f_496f_b9b9_40f7da4a0b90.slice/crio-a44a89496155083559f635b2c86400fc70cdd60cbda09e6eae80001edd071cc8 WatchSource:0}: Error finding container a44a89496155083559f635b2c86400fc70cdd60cbda09e6eae80001edd071cc8: Status 404 returned error can't find the container with id a44a89496155083559f635b2c86400fc70cdd60cbda09e6eae80001edd071cc8 Jul 07 00:03:25.020889 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:25.020851 2562 generic.go:356] "Generic (PLEG): container finished" podID="f8441d57-440f-496f-b9b9-40f7da4a0b90" containerID="32b79ed8e5cebe6314a6eae161e319acb97ace99fce8dba85665f1ede70448c7" exitCode=0 Jul 07 00:03:25.021109 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:25.020914 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" event={"ID":"f8441d57-440f-496f-b9b9-40f7da4a0b90","Type":"ContainerDied","Data":"32b79ed8e5cebe6314a6eae161e319acb97ace99fce8dba85665f1ede70448c7"} Jul 07 00:03:25.021109 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:25.020948 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" event={"ID":"f8441d57-440f-496f-b9b9-40f7da4a0b90","Type":"ContainerStarted","Data":"a44a89496155083559f635b2c86400fc70cdd60cbda09e6eae80001edd071cc8"} Jul 07 00:03:26.026791 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:26.026755 2562 generic.go:356] "Generic (PLEG): container finished" podID="f8441d57-440f-496f-b9b9-40f7da4a0b90" containerID="c79c38dad2c9d2aeaf52cd23affdf1f3af7bf1b883a47efce898e0b26022f7f0" exitCode=0 Jul 07 00:03:26.027265 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:26.026814 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" event={"ID":"f8441d57-440f-496f-b9b9-40f7da4a0b90","Type":"ContainerDied","Data":"c79c38dad2c9d2aeaf52cd23affdf1f3af7bf1b883a47efce898e0b26022f7f0"} Jul 07 00:03:27.033420 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:27.033387 2562 generic.go:356] "Generic (PLEG): container finished" podID="f8441d57-440f-496f-b9b9-40f7da4a0b90" containerID="41455de94a8df7e3472230dd4f7b85348d4ea4b7573d405cfea67ddbc2852707" exitCode=0 Jul 07 00:03:27.033822 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:27.033467 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" event={"ID":"f8441d57-440f-496f-b9b9-40f7da4a0b90","Type":"ContainerDied","Data":"41455de94a8df7e3472230dd4f7b85348d4ea4b7573d405cfea67ddbc2852707"} Jul 07 00:03:28.164741 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.164711 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:28.190199 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.190168 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgsbl\" (UniqueName: \"kubernetes.io/projected/f8441d57-440f-496f-b9b9-40f7da4a0b90-kube-api-access-pgsbl\") pod \"f8441d57-440f-496f-b9b9-40f7da4a0b90\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " Jul 07 00:03:28.190354 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.190247 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-util\") pod \"f8441d57-440f-496f-b9b9-40f7da4a0b90\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " Jul 07 00:03:28.190354 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.190278 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-bundle\") pod \"f8441d57-440f-496f-b9b9-40f7da4a0b90\" (UID: \"f8441d57-440f-496f-b9b9-40f7da4a0b90\") " Jul 07 00:03:28.191044 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.191013 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-bundle" (OuterVolumeSpecName: "bundle") pod "f8441d57-440f-496f-b9b9-40f7da4a0b90" (UID: "f8441d57-440f-496f-b9b9-40f7da4a0b90"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:03:28.192524 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.192503 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8441d57-440f-496f-b9b9-40f7da4a0b90-kube-api-access-pgsbl" (OuterVolumeSpecName: "kube-api-access-pgsbl") pod "f8441d57-440f-496f-b9b9-40f7da4a0b90" (UID: "f8441d57-440f-496f-b9b9-40f7da4a0b90"). InnerVolumeSpecName "kube-api-access-pgsbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:03:28.196435 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.196393 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-util" (OuterVolumeSpecName: "util") pod "f8441d57-440f-496f-b9b9-40f7da4a0b90" (UID: "f8441d57-440f-496f-b9b9-40f7da4a0b90"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:03:28.290949 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.290860 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-util\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:03:28.290949 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.290889 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8441d57-440f-496f-b9b9-40f7da4a0b90-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:03:28.290949 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.290898 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgsbl\" (UniqueName: \"kubernetes.io/projected/f8441d57-440f-496f-b9b9-40f7da4a0b90-kube-api-access-pgsbl\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:03:28.759529 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.759495 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4"] Jul 07 00:03:28.759777 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.759763 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8441d57-440f-496f-b9b9-40f7da4a0b90" containerName="util" Jul 07 00:03:28.759777 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.759778 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8441d57-440f-496f-b9b9-40f7da4a0b90" containerName="util" Jul 07 00:03:28.759860 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.759796 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8441d57-440f-496f-b9b9-40f7da4a0b90" containerName="extract" Jul 07 00:03:28.759860 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.759801 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8441d57-440f-496f-b9b9-40f7da4a0b90" containerName="extract" Jul 07 00:03:28.759860 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.759810 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8441d57-440f-496f-b9b9-40f7da4a0b90" containerName="pull" Jul 07 00:03:28.759860 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.759815 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8441d57-440f-496f-b9b9-40f7da4a0b90" containerName="pull" Jul 07 00:03:28.759860 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.759856 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8441d57-440f-496f-b9b9-40f7da4a0b90" containerName="extract" Jul 07 00:03:28.762783 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.762764 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.765666 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.765642 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-255ns\"" Jul 07 00:03:28.766839 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.766809 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 00:03:28.766951 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.766813 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 00:03:28.766951 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.766813 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 00:03:28.766951 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.766879 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 00:03:28.771105 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.771087 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 00:03:28.788954 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.788922 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4"] Jul 07 00:03:28.793627 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.793599 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-metrics-cert\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.793754 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.793632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-manager-config\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.793754 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.793681 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-cert\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.793754 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.793734 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdnmd\" (UniqueName: \"kubernetes.io/projected/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-kube-api-access-pdnmd\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.894907 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.894868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-cert\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.894907 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.894913 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdnmd\" (UniqueName: \"kubernetes.io/projected/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-kube-api-access-pdnmd\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.895232 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.894957 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-metrics-cert\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.895232 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.894976 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-manager-config\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.895779 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.895750 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-manager-config\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.897623 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.897596 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-metrics-cert\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.897702 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.897675 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-cert\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:28.910981 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:28.910949 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdnmd\" (UniqueName: \"kubernetes.io/projected/0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3-kube-api-access-pdnmd\") pod \"lws-controller-manager-5df8c9476f-mnjh4\" (UID: \"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3\") " pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:29.043426 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:29.043400 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" Jul 07 00:03:29.043592 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:29.043423 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpvrxk" event={"ID":"f8441d57-440f-496f-b9b9-40f7da4a0b90","Type":"ContainerDied","Data":"a44a89496155083559f635b2c86400fc70cdd60cbda09e6eae80001edd071cc8"} Jul 07 00:03:29.043592 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:29.043468 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a44a89496155083559f635b2c86400fc70cdd60cbda09e6eae80001edd071cc8" Jul 07 00:03:29.072589 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:29.072547 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:29.229696 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:29.229659 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4"] Jul 07 00:03:29.240028 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:03:29.239995 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae3b8ac_6c91_4490_bc07_ff1a7622f8a3.slice/crio-5470f245ae20d330fa286bb4356a400771840c1a91b6bb0e445dfea049f7a788 WatchSource:0}: Error finding container 5470f245ae20d330fa286bb4356a400771840c1a91b6bb0e445dfea049f7a788: Status 404 returned error can't find the container with id 5470f245ae20d330fa286bb4356a400771840c1a91b6bb0e445dfea049f7a788 Jul 07 00:03:30.051692 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:30.051644 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" event={"ID":"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3","Type":"ContainerStarted","Data":"5470f245ae20d330fa286bb4356a400771840c1a91b6bb0e445dfea049f7a788"} Jul 07 00:03:31.057423 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:31.057321 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" event={"ID":"0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3","Type":"ContainerStarted","Data":"f8bddc5b36f22038349c80321cdccaded639571eb2aa683876a11c8f3fb8268c"} Jul 07 00:03:31.057423 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:31.057388 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:31.077843 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:31.077784 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" podStartSLOduration=1.589455374 podStartE2EDuration="3.077764606s" totalImagesPullingTime="1.488309232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:31.076481496 +0000 UTC m=+377.048966158" watchObservedRunningTime="2026-07-07 00:03:31.077764606 +0000 UTC m=+377.050249266" Jul 07 00:03:42.015132 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.015088 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc"] Jul 07 00:03:42.069296 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.069257 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5df8c9476f-mnjh4" Jul 07 00:03:42.069485 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.069435 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.070039 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.070013 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc"] Jul 07 00:03:42.075088 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.075045 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 00:03:42.075923 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.075895 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dmrqj\"" Jul 07 00:03:42.076418 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.076395 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 00:03:42.077975 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.077955 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 00:03:42.080627 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.080611 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 00:03:42.092397 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.092367 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09ebb26f-ac7b-4a20-be81-608ac47a4038-tmp\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.092578 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.092446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/09ebb26f-ac7b-4a20-be81-608ac47a4038-manifests\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.092578 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.092494 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/09ebb26f-ac7b-4a20-be81-608ac47a4038-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.092578 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.092530 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wzj9\" (UniqueName: \"kubernetes.io/projected/09ebb26f-ac7b-4a20-be81-608ac47a4038-kube-api-access-5wzj9\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.092745 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.092617 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/09ebb26f-ac7b-4a20-be81-608ac47a4038-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.193995 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.193962 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09ebb26f-ac7b-4a20-be81-608ac47a4038-tmp\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.193995 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.194008 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/09ebb26f-ac7b-4a20-be81-608ac47a4038-manifests\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.194277 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.194133 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/09ebb26f-ac7b-4a20-be81-608ac47a4038-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.194277 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.194172 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wzj9\" (UniqueName: \"kubernetes.io/projected/09ebb26f-ac7b-4a20-be81-608ac47a4038-kube-api-access-5wzj9\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.194277 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.194208 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/09ebb26f-ac7b-4a20-be81-608ac47a4038-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.194433 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.194367 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/09ebb26f-ac7b-4a20-be81-608ac47a4038-manifests\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.196444 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.196411 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09ebb26f-ac7b-4a20-be81-608ac47a4038-tmp\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.196619 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.196491 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/09ebb26f-ac7b-4a20-be81-608ac47a4038-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.196619 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.196578 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/09ebb26f-ac7b-4a20-be81-608ac47a4038-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.218676 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.218639 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wzj9\" (UniqueName: \"kubernetes.io/projected/09ebb26f-ac7b-4a20-be81-608ac47a4038-kube-api-access-5wzj9\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-4trmc\" (UID: \"09ebb26f-ac7b-4a20-be81-608ac47a4038\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.381139 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.381096 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:42.584542 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:42.584507 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc"] Jul 07 00:03:42.594496 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:03:42.594466 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09ebb26f_ac7b_4a20_be81_608ac47a4038.slice/crio-a8a9b556fbc6c9d3e5fbdbada1061dfa961326de389b0a703eea8a09fb20fcf4 WatchSource:0}: Error finding container a8a9b556fbc6c9d3e5fbdbada1061dfa961326de389b0a703eea8a09fb20fcf4: Status 404 returned error can't find the container with id a8a9b556fbc6c9d3e5fbdbada1061dfa961326de389b0a703eea8a09fb20fcf4 Jul 07 00:03:43.110121 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:43.110079 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" event={"ID":"09ebb26f-ac7b-4a20-be81-608ac47a4038","Type":"ContainerStarted","Data":"a8a9b556fbc6c9d3e5fbdbada1061dfa961326de389b0a703eea8a09fb20fcf4"} Jul 07 00:03:46.127752 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:46.127710 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" event={"ID":"09ebb26f-ac7b-4a20-be81-608ac47a4038","Type":"ContainerStarted","Data":"69ad8b22d5c205aba9d0b7cb5245ff18bb7651b9a0f4157a3fe936859253b958"} Jul 07 00:03:47.133719 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:47.133686 2562 generic.go:356] "Generic (PLEG): container finished" podID="09ebb26f-ac7b-4a20-be81-608ac47a4038" containerID="69ad8b22d5c205aba9d0b7cb5245ff18bb7651b9a0f4157a3fe936859253b958" exitCode=0 Jul 07 00:03:47.134146 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:47.133767 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" event={"ID":"09ebb26f-ac7b-4a20-be81-608ac47a4038","Type":"ContainerDied","Data":"69ad8b22d5c205aba9d0b7cb5245ff18bb7651b9a0f4157a3fe936859253b958"} Jul 07 00:03:49.143663 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:49.143630 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" event={"ID":"09ebb26f-ac7b-4a20-be81-608ac47a4038","Type":"ContainerStarted","Data":"94a1fcbac2200b1d0f1c192e6e590ccd2d02898c808e4187f389cabb2f481290"} Jul 07 00:03:49.144163 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:49.143728 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:03:49.172738 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:03:49.172676 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" podStartSLOduration=3.250810075 podStartE2EDuration="8.172660107s" totalImagesPullingTime="3.888811157s" totalInitContainerRuntime="1.033038875s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:49.169723008 +0000 UTC m=+395.142207663" watchObservedRunningTime="2026-07-07 00:03:49.172660107 +0000 UTC m=+395.145144768" Jul 07 00:04:00.151145 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:00.151108 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-4trmc" Jul 07 00:04:10.841012 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.840976 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c"] Jul 07 00:04:10.844155 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.844137 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:10.847851 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.847830 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 00:04:10.848011 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.847994 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 00:04:10.848084 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.848024 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 00:04:10.848831 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.848812 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c2d9q\"" Jul 07 00:04:10.848921 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.848833 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 00:04:10.849194 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.849176 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 00:04:10.849299 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.849277 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 00:04:10.859164 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.859142 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c"] Jul 07 00:04:10.911902 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.911870 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:10.912101 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.911918 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b56aae5-875d-4c1f-82e7-70e360f432b3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:10.912101 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.911937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:10.912101 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.911997 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxwv7\" (UniqueName: \"kubernetes.io/projected/2b56aae5-875d-4c1f-82e7-70e360f432b3-kube-api-access-jxwv7\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:10.912101 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.912045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2b56aae5-875d-4c1f-82e7-70e360f432b3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:10.912101 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.912097 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:10.912306 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:10.912123 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.012640 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.012603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.012835 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.012651 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b56aae5-875d-4c1f-82e7-70e360f432b3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.012835 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.012671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.012835 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.012707 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxwv7\" (UniqueName: \"kubernetes.io/projected/2b56aae5-875d-4c1f-82e7-70e360f432b3-kube-api-access-jxwv7\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.012835 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.012743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2b56aae5-875d-4c1f-82e7-70e360f432b3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.012835 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.012781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.012835 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.012798 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.013372 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.013341 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.015247 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.015080 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b56aae5-875d-4c1f-82e7-70e360f432b3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.015648 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.015627 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.015814 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.015663 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2b56aae5-875d-4c1f-82e7-70e360f432b3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.015910 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.015681 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.022041 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.022015 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxwv7\" (UniqueName: \"kubernetes.io/projected/2b56aae5-875d-4c1f-82e7-70e360f432b3-kube-api-access-jxwv7\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.022169 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.022125 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b56aae5-875d-4c1f-82e7-70e360f432b3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-59f7c\" (UID: \"2b56aae5-875d-4c1f-82e7-70e360f432b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.153366 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.153327 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:11.327029 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:11.326994 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c"] Jul 07 00:04:11.343321 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:04:11.343285 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b56aae5_875d_4c1f_82e7_70e360f432b3.slice/crio-fc35e803a95cad0eee11c2f2964342e6d485142a0198acafc037e6618ee97aec WatchSource:0}: Error finding container fc35e803a95cad0eee11c2f2964342e6d485142a0198acafc037e6618ee97aec: Status 404 returned error can't find the container with id fc35e803a95cad0eee11c2f2964342e6d485142a0198acafc037e6618ee97aec Jul 07 00:04:12.242128 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:12.242087 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" event={"ID":"2b56aae5-875d-4c1f-82e7-70e360f432b3","Type":"ContainerStarted","Data":"fc35e803a95cad0eee11c2f2964342e6d485142a0198acafc037e6618ee97aec"} Jul 07 00:04:14.389545 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:14.389498 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:04:14.389833 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:14.389585 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:04:15.258197 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:15.258162 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" event={"ID":"2b56aae5-875d-4c1f-82e7-70e360f432b3","Type":"ContainerStarted","Data":"60e2a5734c10e2a33dd85828eeb6c465c5024565a8015775b5a372cf50358b9e"} Jul 07 00:04:15.258367 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:15.258312 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:04:15.259963 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:15.259934 2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-59f7c container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 00:04:15.260100 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:15.259989 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" podUID="2b56aae5-875d-4c1f-82e7-70e360f432b3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 00:04:15.288789 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:15.288719 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" podStartSLOduration=2.244767456 podStartE2EDuration="5.288696368s" totalImagesPullingTime="3.043928912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:15.288401315 +0000 UTC m=+421.260885976" watchObservedRunningTime="2026-07-07 00:04:15.288696368 +0000 UTC m=+421.261181031" Jul 07 00:04:16.263885 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:04:16.263850 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-59f7c" Jul 07 00:05:18.995331 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:18.995290 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6"] Jul 07 00:05:18.998477 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:18.998460 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:19.000987 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.000960 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 00:05:19.001142 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.000960 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 00:05:19.001905 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.001884 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v659t\"" Jul 07 00:05:19.006347 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.006322 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6"] Jul 07 00:05:19.083208 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.083169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:19.083399 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.083224 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8w2j\" (UniqueName: \"kubernetes.io/projected/d63a0916-a7af-4d2b-941d-242ee4704ba0-kube-api-access-q8w2j\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:19.083399 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.083317 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:19.184163 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.184111 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8w2j\" (UniqueName: \"kubernetes.io/projected/d63a0916-a7af-4d2b-941d-242ee4704ba0-kube-api-access-q8w2j\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:19.184359 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.184190 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:19.184359 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.184229 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:19.184579 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.184562 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:19.184658 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.184637 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:19.195642 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.195605 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8w2j\" (UniqueName: \"kubernetes.io/projected/d63a0916-a7af-4d2b-941d-242ee4704ba0-kube-api-access-q8w2j\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:19.309445 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.309350 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:19.401290 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.401250 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf"] Jul 07 00:05:19.405348 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.405326 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:19.411950 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.411924 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf"] Jul 07 00:05:19.451376 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.451353 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6"] Jul 07 00:05:19.453387 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:05:19.453356 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd63a0916_a7af_4d2b_941d_242ee4704ba0.slice/crio-42193c3fcbad7255819de49e21bac54c851dd06ac4c85dc9f401194a95c49b16 WatchSource:0}: Error finding container 42193c3fcbad7255819de49e21bac54c851dd06ac4c85dc9f401194a95c49b16: Status 404 returned error can't find the container with id 42193c3fcbad7255819de49e21bac54c851dd06ac4c85dc9f401194a95c49b16 Jul 07 00:05:19.487079 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.487040 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv7hg\" (UniqueName: \"kubernetes.io/projected/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-kube-api-access-mv7hg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:19.487180 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.487144 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:19.487226 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.487218 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:19.540650 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.540611 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" event={"ID":"d63a0916-a7af-4d2b-941d-242ee4704ba0","Type":"ContainerStarted","Data":"0fe0efaf7583daf3f729cdecad94eec9a72e6dc6756bea9256cc9d1380412b2f"} Jul 07 00:05:19.540827 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.540665 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" event={"ID":"d63a0916-a7af-4d2b-941d-242ee4704ba0","Type":"ContainerStarted","Data":"42193c3fcbad7255819de49e21bac54c851dd06ac4c85dc9f401194a95c49b16"} Jul 07 00:05:19.588517 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.588423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:19.588517 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.588476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv7hg\" (UniqueName: \"kubernetes.io/projected/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-kube-api-access-mv7hg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:19.588517 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.588509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:19.588843 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.588824 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:19.588882 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.588842 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:19.596959 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.596937 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv7hg\" (UniqueName: \"kubernetes.io/projected/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-kube-api-access-mv7hg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:19.718051 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.718012 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:19.795124 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.795095 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz"] Jul 07 00:05:19.799167 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.798869 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:19.806860 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.806559 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz"] Jul 07 00:05:19.854491 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.854466 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf"] Jul 07 00:05:19.857854 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:05:19.857826 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded65c1a6_9419_47ac_ba01_8d7dc007b18f.slice/crio-3d1851df29cf42f68d45c8b1f10585ce58531ad4688b53ac960c7bb778bb9fe9 WatchSource:0}: Error finding container 3d1851df29cf42f68d45c8b1f10585ce58531ad4688b53ac960c7bb778bb9fe9: Status 404 returned error can't find the container with id 3d1851df29cf42f68d45c8b1f10585ce58531ad4688b53ac960c7bb778bb9fe9 Jul 07 00:05:19.890046 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.890016 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:19.890181 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.890095 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr68g\" (UniqueName: \"kubernetes.io/projected/4c1fb982-b6f5-4687-98c5-e6207076c890-kube-api-access-xr68g\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:19.890227 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.890189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:19.991448 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.991409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr68g\" (UniqueName: \"kubernetes.io/projected/4c1fb982-b6f5-4687-98c5-e6207076c890-kube-api-access-xr68g\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:19.991629 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.991470 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:19.991629 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.991523 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:19.991868 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.991852 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:19.991913 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.991895 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:19.999687 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:19.999664 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr68g\" (UniqueName: \"kubernetes.io/projected/4c1fb982-b6f5-4687-98c5-e6207076c890-kube-api-access-xr68g\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:20.111919 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.111824 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:20.197080 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.196813 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c"] Jul 07 00:05:20.208574 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.208524 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:20.215469 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.215441 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c"] Jul 07 00:05:20.253269 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.253244 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz"] Jul 07 00:05:20.255572 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:05:20.255545 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c1fb982_b6f5_4687_98c5_e6207076c890.slice/crio-64994450087c49e6b2e28fef75a5edec87fa2c248c3e014a20d5cc924aa7a704 WatchSource:0}: Error finding container 64994450087c49e6b2e28fef75a5edec87fa2c248c3e014a20d5cc924aa7a704: Status 404 returned error can't find the container with id 64994450087c49e6b2e28fef75a5edec87fa2c248c3e014a20d5cc924aa7a704 Jul 07 00:05:20.294921 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.294890 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:20.295119 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.294936 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jjwp\" (UniqueName: \"kubernetes.io/projected/4c2be1b3-a140-4e94-8981-b9f8d257846f-kube-api-access-6jjwp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:20.295119 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.295041 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:20.395816 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.395781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:20.395816 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.395824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jjwp\" (UniqueName: \"kubernetes.io/projected/4c2be1b3-a140-4e94-8981-b9f8d257846f-kube-api-access-6jjwp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:20.396089 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.395867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:20.396257 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.396234 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:20.396317 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.396287 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:20.404202 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.404181 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jjwp\" (UniqueName: \"kubernetes.io/projected/4c2be1b3-a140-4e94-8981-b9f8d257846f-kube-api-access-6jjwp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:20.526562 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.526518 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:20.546254 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.546221 2562 generic.go:356] "Generic (PLEG): container finished" podID="d63a0916-a7af-4d2b-941d-242ee4704ba0" containerID="0fe0efaf7583daf3f729cdecad94eec9a72e6dc6756bea9256cc9d1380412b2f" exitCode=0 Jul 07 00:05:20.546419 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.546295 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" event={"ID":"d63a0916-a7af-4d2b-941d-242ee4704ba0","Type":"ContainerDied","Data":"0fe0efaf7583daf3f729cdecad94eec9a72e6dc6756bea9256cc9d1380412b2f"} Jul 07 00:05:20.547975 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.547948 2562 generic.go:356] "Generic (PLEG): container finished" podID="4c1fb982-b6f5-4687-98c5-e6207076c890" containerID="598168e91f5796b9fd3f5b9b41c295bf988d39721224df0e9fa8774876ef0ce9" exitCode=0 Jul 07 00:05:20.548107 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.548022 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" event={"ID":"4c1fb982-b6f5-4687-98c5-e6207076c890","Type":"ContainerDied","Data":"598168e91f5796b9fd3f5b9b41c295bf988d39721224df0e9fa8774876ef0ce9"} Jul 07 00:05:20.548107 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.548097 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" event={"ID":"4c1fb982-b6f5-4687-98c5-e6207076c890","Type":"ContainerStarted","Data":"64994450087c49e6b2e28fef75a5edec87fa2c248c3e014a20d5cc924aa7a704"} Jul 07 00:05:20.550222 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.550191 2562 generic.go:356] "Generic (PLEG): container finished" podID="ed65c1a6-9419-47ac-ba01-8d7dc007b18f" containerID="9d34750280e3f7a61b425548a50066c7a46ec826fe67389163d6ab57973ba56c" exitCode=0 Jul 07 00:05:20.550340 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.550244 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" event={"ID":"ed65c1a6-9419-47ac-ba01-8d7dc007b18f","Type":"ContainerDied","Data":"9d34750280e3f7a61b425548a50066c7a46ec826fe67389163d6ab57973ba56c"} Jul 07 00:05:20.550340 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.550297 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" event={"ID":"ed65c1a6-9419-47ac-ba01-8d7dc007b18f","Type":"ContainerStarted","Data":"3d1851df29cf42f68d45c8b1f10585ce58531ad4688b53ac960c7bb778bb9fe9"} Jul 07 00:05:20.663779 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:20.663746 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c"] Jul 07 00:05:20.666795 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:05:20.666769 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c2be1b3_a140_4e94_8981_b9f8d257846f.slice/crio-c6997e79ed92301122a5f63b2ff7927b3275e97b15252940c2e8b4713d3e6f49 WatchSource:0}: Error finding container c6997e79ed92301122a5f63b2ff7927b3275e97b15252940c2e8b4713d3e6f49: Status 404 returned error can't find the container with id c6997e79ed92301122a5f63b2ff7927b3275e97b15252940c2e8b4713d3e6f49 Jul 07 00:05:21.558732 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:21.558700 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" event={"ID":"d63a0916-a7af-4d2b-941d-242ee4704ba0","Type":"ContainerStarted","Data":"4f01e467c5b735dc4efeb7dcaf7dc0d29c867a15cebe82910e39debb37d039e4"} Jul 07 00:05:21.561206 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:21.561172 2562 generic.go:356] "Generic (PLEG): container finished" podID="4c2be1b3-a140-4e94-8981-b9f8d257846f" containerID="cdfe856c7defda0690bdc0f023cb774901b8ccf10e773a0d25414245d8797406" exitCode=0 Jul 07 00:05:21.561356 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:21.561271 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" event={"ID":"4c2be1b3-a140-4e94-8981-b9f8d257846f","Type":"ContainerDied","Data":"cdfe856c7defda0690bdc0f023cb774901b8ccf10e773a0d25414245d8797406"} Jul 07 00:05:21.561356 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:21.561340 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" event={"ID":"4c2be1b3-a140-4e94-8981-b9f8d257846f","Type":"ContainerStarted","Data":"c6997e79ed92301122a5f63b2ff7927b3275e97b15252940c2e8b4713d3e6f49"} Jul 07 00:05:21.563811 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:21.563784 2562 generic.go:356] "Generic (PLEG): container finished" podID="ed65c1a6-9419-47ac-ba01-8d7dc007b18f" containerID="c2cf8d712714f4db9bd3c9df69267a3bc29cca706ca5f3c80d667ddb4ca7f0ae" exitCode=0 Jul 07 00:05:21.563927 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:21.563823 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" event={"ID":"ed65c1a6-9419-47ac-ba01-8d7dc007b18f","Type":"ContainerDied","Data":"c2cf8d712714f4db9bd3c9df69267a3bc29cca706ca5f3c80d667ddb4ca7f0ae"} Jul 07 00:05:22.570558 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:22.570524 2562 generic.go:356] "Generic (PLEG): container finished" podID="4c1fb982-b6f5-4687-98c5-e6207076c890" containerID="4abdd05961f51ec745f8602d9bafa65fcccb3e8ef65be475c6c553c56c8978a7" exitCode=0 Jul 07 00:05:22.570953 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:22.570638 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" event={"ID":"4c1fb982-b6f5-4687-98c5-e6207076c890","Type":"ContainerDied","Data":"4abdd05961f51ec745f8602d9bafa65fcccb3e8ef65be475c6c553c56c8978a7"} Jul 07 00:05:22.573337 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:22.573305 2562 generic.go:356] "Generic (PLEG): container finished" podID="ed65c1a6-9419-47ac-ba01-8d7dc007b18f" containerID="7368b7b5df5d870aab10edb14d696bb59a8303e17ec6d040d5d77a054bb8428d" exitCode=0 Jul 07 00:05:22.573498 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:22.573360 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" event={"ID":"ed65c1a6-9419-47ac-ba01-8d7dc007b18f","Type":"ContainerDied","Data":"7368b7b5df5d870aab10edb14d696bb59a8303e17ec6d040d5d77a054bb8428d"} Jul 07 00:05:22.575448 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:22.575423 2562 generic.go:356] "Generic (PLEG): container finished" podID="d63a0916-a7af-4d2b-941d-242ee4704ba0" containerID="4f01e467c5b735dc4efeb7dcaf7dc0d29c867a15cebe82910e39debb37d039e4" exitCode=0 Jul 07 00:05:22.575547 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:22.575459 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" event={"ID":"d63a0916-a7af-4d2b-941d-242ee4704ba0","Type":"ContainerDied","Data":"4f01e467c5b735dc4efeb7dcaf7dc0d29c867a15cebe82910e39debb37d039e4"} Jul 07 00:05:23.582173 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.582059 2562 generic.go:356] "Generic (PLEG): container finished" podID="4c1fb982-b6f5-4687-98c5-e6207076c890" containerID="fb978f0f214c4f3ec23404513a337704d7854ea2c2557b8eb0d828b3dc164171" exitCode=0 Jul 07 00:05:23.582173 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.582111 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" event={"ID":"4c1fb982-b6f5-4687-98c5-e6207076c890","Type":"ContainerDied","Data":"fb978f0f214c4f3ec23404513a337704d7854ea2c2557b8eb0d828b3dc164171"} Jul 07 00:05:23.584002 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.583974 2562 generic.go:356] "Generic (PLEG): container finished" podID="4c2be1b3-a140-4e94-8981-b9f8d257846f" containerID="ecd63d2f57149771f66f77f42131ad66777fd7bec758e1cd17d56faeeea467d7" exitCode=0 Jul 07 00:05:23.584121 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.584059 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" event={"ID":"4c2be1b3-a140-4e94-8981-b9f8d257846f","Type":"ContainerDied","Data":"ecd63d2f57149771f66f77f42131ad66777fd7bec758e1cd17d56faeeea467d7"} Jul 07 00:05:23.586377 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.586354 2562 generic.go:356] "Generic (PLEG): container finished" podID="d63a0916-a7af-4d2b-941d-242ee4704ba0" containerID="0e650d43f4c2755b15236c68e63a1c656d36bf66831ce320ca9e60a7c95c5c02" exitCode=0 Jul 07 00:05:23.586505 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.586434 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" event={"ID":"d63a0916-a7af-4d2b-941d-242ee4704ba0","Type":"ContainerDied","Data":"0e650d43f4c2755b15236c68e63a1c656d36bf66831ce320ca9e60a7c95c5c02"} Jul 07 00:05:23.729472 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.729450 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:23.833567 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.833471 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv7hg\" (UniqueName: \"kubernetes.io/projected/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-kube-api-access-mv7hg\") pod \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " Jul 07 00:05:23.833567 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.833538 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-bundle\") pod \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " Jul 07 00:05:23.833755 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.833591 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-util\") pod \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\" (UID: \"ed65c1a6-9419-47ac-ba01-8d7dc007b18f\") " Jul 07 00:05:23.834280 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.834243 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-bundle" (OuterVolumeSpecName: "bundle") pod "ed65c1a6-9419-47ac-ba01-8d7dc007b18f" (UID: "ed65c1a6-9419-47ac-ba01-8d7dc007b18f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:05:23.835828 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.835806 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-kube-api-access-mv7hg" (OuterVolumeSpecName: "kube-api-access-mv7hg") pod "ed65c1a6-9419-47ac-ba01-8d7dc007b18f" (UID: "ed65c1a6-9419-47ac-ba01-8d7dc007b18f"). InnerVolumeSpecName "kube-api-access-mv7hg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:05:23.838830 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.838804 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-util" (OuterVolumeSpecName: "util") pod "ed65c1a6-9419-47ac-ba01-8d7dc007b18f" (UID: "ed65c1a6-9419-47ac-ba01-8d7dc007b18f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:05:23.934705 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.934663 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv7hg\" (UniqueName: \"kubernetes.io/projected/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-kube-api-access-mv7hg\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:23.934705 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.934697 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:23.934705 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:23.934707 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed65c1a6-9419-47ac-ba01-8d7dc007b18f-util\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:24.593760 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.593729 2562 generic.go:356] "Generic (PLEG): container finished" podID="4c2be1b3-a140-4e94-8981-b9f8d257846f" containerID="09061cc6340536173865b64f933f76c9c299063e7f1ec7104a48ec3ccbfaef4a" exitCode=0 Jul 07 00:05:24.594133 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.593802 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" event={"ID":"4c2be1b3-a140-4e94-8981-b9f8d257846f","Type":"ContainerDied","Data":"09061cc6340536173865b64f933f76c9c299063e7f1ec7104a48ec3ccbfaef4a"} Jul 07 00:05:24.595751 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.595724 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" Jul 07 00:05:24.595880 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.595765 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf" event={"ID":"ed65c1a6-9419-47ac-ba01-8d7dc007b18f","Type":"ContainerDied","Data":"3d1851df29cf42f68d45c8b1f10585ce58531ad4688b53ac960c7bb778bb9fe9"} Jul 07 00:05:24.595880 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.595842 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d1851df29cf42f68d45c8b1f10585ce58531ad4688b53ac960c7bb778bb9fe9" Jul 07 00:05:24.733385 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.733356 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:24.768885 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.768862 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:24.842428 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.842338 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-util\") pod \"d63a0916-a7af-4d2b-941d-242ee4704ba0\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " Jul 07 00:05:24.842428 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.842393 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-bundle\") pod \"d63a0916-a7af-4d2b-941d-242ee4704ba0\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " Jul 07 00:05:24.842662 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.842435 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr68g\" (UniqueName: \"kubernetes.io/projected/4c1fb982-b6f5-4687-98c5-e6207076c890-kube-api-access-xr68g\") pod \"4c1fb982-b6f5-4687-98c5-e6207076c890\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " Jul 07 00:05:24.842662 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.842484 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-bundle\") pod \"4c1fb982-b6f5-4687-98c5-e6207076c890\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " Jul 07 00:05:24.842662 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.842505 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-util\") pod \"4c1fb982-b6f5-4687-98c5-e6207076c890\" (UID: \"4c1fb982-b6f5-4687-98c5-e6207076c890\") " Jul 07 00:05:24.842662 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.842554 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8w2j\" (UniqueName: \"kubernetes.io/projected/d63a0916-a7af-4d2b-941d-242ee4704ba0-kube-api-access-q8w2j\") pod \"d63a0916-a7af-4d2b-941d-242ee4704ba0\" (UID: \"d63a0916-a7af-4d2b-941d-242ee4704ba0\") " Jul 07 00:05:24.843023 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.842999 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-bundle" (OuterVolumeSpecName: "bundle") pod "d63a0916-a7af-4d2b-941d-242ee4704ba0" (UID: "d63a0916-a7af-4d2b-941d-242ee4704ba0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:05:24.843183 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.843153 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-bundle" (OuterVolumeSpecName: "bundle") pod "4c1fb982-b6f5-4687-98c5-e6207076c890" (UID: "4c1fb982-b6f5-4687-98c5-e6207076c890"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:05:24.844667 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.844642 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c1fb982-b6f5-4687-98c5-e6207076c890-kube-api-access-xr68g" (OuterVolumeSpecName: "kube-api-access-xr68g") pod "4c1fb982-b6f5-4687-98c5-e6207076c890" (UID: "4c1fb982-b6f5-4687-98c5-e6207076c890"). InnerVolumeSpecName "kube-api-access-xr68g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:05:24.844755 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.844696 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d63a0916-a7af-4d2b-941d-242ee4704ba0-kube-api-access-q8w2j" (OuterVolumeSpecName: "kube-api-access-q8w2j") pod "d63a0916-a7af-4d2b-941d-242ee4704ba0" (UID: "d63a0916-a7af-4d2b-941d-242ee4704ba0"). InnerVolumeSpecName "kube-api-access-q8w2j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:05:24.849047 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.849022 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-util" (OuterVolumeSpecName: "util") pod "d63a0916-a7af-4d2b-941d-242ee4704ba0" (UID: "d63a0916-a7af-4d2b-941d-242ee4704ba0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:05:24.850098 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.850062 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-util" (OuterVolumeSpecName: "util") pod "4c1fb982-b6f5-4687-98c5-e6207076c890" (UID: "4c1fb982-b6f5-4687-98c5-e6207076c890"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:05:24.943609 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.943561 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-util\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:24.943609 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.943603 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d63a0916-a7af-4d2b-941d-242ee4704ba0-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:24.943609 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.943613 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr68g\" (UniqueName: \"kubernetes.io/projected/4c1fb982-b6f5-4687-98c5-e6207076c890-kube-api-access-xr68g\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:24.943609 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.943622 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:24.943609 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.943631 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c1fb982-b6f5-4687-98c5-e6207076c890-util\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:24.943925 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:24.943639 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8w2j\" (UniqueName: \"kubernetes.io/projected/d63a0916-a7af-4d2b-941d-242ee4704ba0-kube-api-access-q8w2j\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:25.602200 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.602092 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" event={"ID":"d63a0916-a7af-4d2b-941d-242ee4704ba0","Type":"ContainerDied","Data":"42193c3fcbad7255819de49e21bac54c851dd06ac4c85dc9f401194a95c49b16"} Jul 07 00:05:25.602200 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.602160 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42193c3fcbad7255819de49e21bac54c851dd06ac4c85dc9f401194a95c49b16" Jul 07 00:05:25.602200 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.602108 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6" Jul 07 00:05:25.604019 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.603996 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" Jul 07 00:05:25.604019 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.604006 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz" event={"ID":"4c1fb982-b6f5-4687-98c5-e6207076c890","Type":"ContainerDied","Data":"64994450087c49e6b2e28fef75a5edec87fa2c248c3e014a20d5cc924aa7a704"} Jul 07 00:05:25.604185 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.604049 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64994450087c49e6b2e28fef75a5edec87fa2c248c3e014a20d5cc924aa7a704" Jul 07 00:05:25.741391 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.741362 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:05:25.851858 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.851816 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-util\") pod \"4c2be1b3-a140-4e94-8981-b9f8d257846f\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " Jul 07 00:05:25.851858 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.851861 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jjwp\" (UniqueName: \"kubernetes.io/projected/4c2be1b3-a140-4e94-8981-b9f8d257846f-kube-api-access-6jjwp\") pod \"4c2be1b3-a140-4e94-8981-b9f8d257846f\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " Jul 07 00:05:25.852146 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.851908 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-bundle\") pod \"4c2be1b3-a140-4e94-8981-b9f8d257846f\" (UID: \"4c2be1b3-a140-4e94-8981-b9f8d257846f\") " Jul 07 00:05:25.852504 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.852444 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-bundle" (OuterVolumeSpecName: "bundle") pod "4c2be1b3-a140-4e94-8981-b9f8d257846f" (UID: "4c2be1b3-a140-4e94-8981-b9f8d257846f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:05:25.854076 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.854034 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c2be1b3-a140-4e94-8981-b9f8d257846f-kube-api-access-6jjwp" (OuterVolumeSpecName: "kube-api-access-6jjwp") pod "4c2be1b3-a140-4e94-8981-b9f8d257846f" (UID: "4c2be1b3-a140-4e94-8981-b9f8d257846f"). InnerVolumeSpecName "kube-api-access-6jjwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:05:25.857376 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.857329 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-util" (OuterVolumeSpecName: "util") pod "4c2be1b3-a140-4e94-8981-b9f8d257846f" (UID: "4c2be1b3-a140-4e94-8981-b9f8d257846f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:05:25.952528 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.952492 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-util\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:25.952528 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.952526 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jjwp\" (UniqueName: \"kubernetes.io/projected/4c2be1b3-a140-4e94-8981-b9f8d257846f-kube-api-access-6jjwp\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:25.952528 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:25.952538 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c2be1b3-a140-4e94-8981-b9f8d257846f-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:05:26.609768 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:26.609731 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" event={"ID":"4c2be1b3-a140-4e94-8981-b9f8d257846f","Type":"ContainerDied","Data":"c6997e79ed92301122a5f63b2ff7927b3275e97b15252940c2e8b4713d3e6f49"} Jul 07 00:05:26.610154 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:26.609784 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6997e79ed92301122a5f63b2ff7927b3275e97b15252940c2e8b4713d3e6f49" Jul 07 00:05:26.610154 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:05:26.609780 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c" Jul 07 00:06:04.425380 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425333 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr"] Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425654 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d63a0916-a7af-4d2b-941d-242ee4704ba0" containerName="pull" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425665 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d63a0916-a7af-4d2b-941d-242ee4704ba0" containerName="pull" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425674 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c1fb982-b6f5-4687-98c5-e6207076c890" containerName="pull" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425679 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c1fb982-b6f5-4687-98c5-e6207076c890" containerName="pull" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425691 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c1fb982-b6f5-4687-98c5-e6207076c890" containerName="util" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425696 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c1fb982-b6f5-4687-98c5-e6207076c890" containerName="util" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425702 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed65c1a6-9419-47ac-ba01-8d7dc007b18f" containerName="extract" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425707 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed65c1a6-9419-47ac-ba01-8d7dc007b18f" containerName="extract" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425713 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c1fb982-b6f5-4687-98c5-e6207076c890" containerName="extract" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425718 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c1fb982-b6f5-4687-98c5-e6207076c890" containerName="extract" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425727 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2be1b3-a140-4e94-8981-b9f8d257846f" containerName="util" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425732 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2be1b3-a140-4e94-8981-b9f8d257846f" containerName="util" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425737 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2be1b3-a140-4e94-8981-b9f8d257846f" containerName="pull" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425742 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2be1b3-a140-4e94-8981-b9f8d257846f" containerName="pull" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425747 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2be1b3-a140-4e94-8981-b9f8d257846f" containerName="extract" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425751 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2be1b3-a140-4e94-8981-b9f8d257846f" containerName="extract" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425758 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d63a0916-a7af-4d2b-941d-242ee4704ba0" containerName="util" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425764 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d63a0916-a7af-4d2b-941d-242ee4704ba0" containerName="util" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425768 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed65c1a6-9419-47ac-ba01-8d7dc007b18f" containerName="util" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425773 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed65c1a6-9419-47ac-ba01-8d7dc007b18f" containerName="util" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425778 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed65c1a6-9419-47ac-ba01-8d7dc007b18f" containerName="pull" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425784 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed65c1a6-9419-47ac-ba01-8d7dc007b18f" containerName="pull" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425790 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d63a0916-a7af-4d2b-941d-242ee4704ba0" containerName="extract" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425796 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d63a0916-a7af-4d2b-941d-242ee4704ba0" containerName="extract" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425843 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c2be1b3-a140-4e94-8981-b9f8d257846f" containerName="extract" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425852 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c1fb982-b6f5-4687-98c5-e6207076c890" containerName="extract" Jul 07 00:06:04.425847 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425859 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d63a0916-a7af-4d2b-941d-242ee4704ba0" containerName="extract" Jul 07 00:06:04.426632 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.425865 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed65c1a6-9419-47ac-ba01-8d7dc007b18f" containerName="extract" Jul 07 00:06:04.430182 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.430165 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:04.433004 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.432975 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 00:06:04.433227 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.433212 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 00:06:04.433312 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.433241 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q4b5q\"" Jul 07 00:06:04.441522 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.441494 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr"] Jul 07 00:06:04.499551 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.499511 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxr6w\" (UniqueName: \"kubernetes.io/projected/9396b799-3109-413a-a21f-0df0b62ef0c5-kube-api-access-cxr6w\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jxphr\" (UID: \"9396b799-3109-413a-a21f-0df0b62ef0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:04.499747 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.499560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9396b799-3109-413a-a21f-0df0b62ef0c5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jxphr\" (UID: \"9396b799-3109-413a-a21f-0df0b62ef0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:04.600426 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.600383 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxr6w\" (UniqueName: \"kubernetes.io/projected/9396b799-3109-413a-a21f-0df0b62ef0c5-kube-api-access-cxr6w\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jxphr\" (UID: \"9396b799-3109-413a-a21f-0df0b62ef0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:04.600426 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.600428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9396b799-3109-413a-a21f-0df0b62ef0c5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jxphr\" (UID: \"9396b799-3109-413a-a21f-0df0b62ef0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:04.600780 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.600764 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9396b799-3109-413a-a21f-0df0b62ef0c5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jxphr\" (UID: \"9396b799-3109-413a-a21f-0df0b62ef0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:04.609272 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.609243 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxr6w\" (UniqueName: \"kubernetes.io/projected/9396b799-3109-413a-a21f-0df0b62ef0c5-kube-api-access-cxr6w\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jxphr\" (UID: \"9396b799-3109-413a-a21f-0df0b62ef0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:04.742118 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.742001 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:04.885662 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:04.885625 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr"] Jul 07 00:06:04.891402 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:06:04.891367 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9396b799_3109_413a_a21f_0df0b62ef0c5.slice/crio-8e88ac323d074498c3e83cf026bf87450362af2478e0f00b86a93867ded570dd WatchSource:0}: Error finding container 8e88ac323d074498c3e83cf026bf87450362af2478e0f00b86a93867ded570dd: Status 404 returned error can't find the container with id 8e88ac323d074498c3e83cf026bf87450362af2478e0f00b86a93867ded570dd Jul 07 00:06:05.104518 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.104492 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr"] Jul 07 00:06:05.119504 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.119470 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr"] Jul 07 00:06:05.128478 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.128448 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs"] Jul 07 00:06:05.133485 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.133459 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:05.152240 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.152199 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs"] Jul 07 00:06:05.205804 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.205769 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2018ca11-5106-4a27-87e3-0d0f58785864-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-646gs\" (UID: \"2018ca11-5106-4a27-87e3-0d0f58785864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:05.205965 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.205817 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tqsp\" (UniqueName: \"kubernetes.io/projected/2018ca11-5106-4a27-87e3-0d0f58785864-kube-api-access-2tqsp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-646gs\" (UID: \"2018ca11-5106-4a27-87e3-0d0f58785864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:05.306958 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.306919 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2018ca11-5106-4a27-87e3-0d0f58785864-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-646gs\" (UID: \"2018ca11-5106-4a27-87e3-0d0f58785864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:05.306958 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.306963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tqsp\" (UniqueName: \"kubernetes.io/projected/2018ca11-5106-4a27-87e3-0d0f58785864-kube-api-access-2tqsp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-646gs\" (UID: \"2018ca11-5106-4a27-87e3-0d0f58785864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:05.307337 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.307318 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2018ca11-5106-4a27-87e3-0d0f58785864-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-646gs\" (UID: \"2018ca11-5106-4a27-87e3-0d0f58785864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:05.318342 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.318314 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tqsp\" (UniqueName: \"kubernetes.io/projected/2018ca11-5106-4a27-87e3-0d0f58785864-kube-api-access-2tqsp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-646gs\" (UID: \"2018ca11-5106-4a27-87e3-0d0f58785864\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:05.444978 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.444883 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:05.584502 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.584463 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs"] Jul 07 00:06:05.588733 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:06:05.588708 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2018ca11_5106_4a27_87e3_0d0f58785864.slice/crio-34ca77135deb139d85e23535f2eb079f212d16a83bd2989efcd89ab9f8e68456 WatchSource:0}: Error finding container 34ca77135deb139d85e23535f2eb079f212d16a83bd2989efcd89ab9f8e68456: Status 404 returned error can't find the container with id 34ca77135deb139d85e23535f2eb079f212d16a83bd2989efcd89ab9f8e68456 Jul 07 00:06:05.794938 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:05.794803 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" event={"ID":"2018ca11-5106-4a27-87e3-0d0f58785864","Type":"ContainerStarted","Data":"34ca77135deb139d85e23535f2eb079f212d16a83bd2989efcd89ab9f8e68456"} Jul 07 00:06:09.820523 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:09.820485 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" event={"ID":"2018ca11-5106-4a27-87e3-0d0f58785864","Type":"ContainerStarted","Data":"179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba"} Jul 07 00:06:09.820965 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:09.820580 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:09.822281 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:09.822246 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" podUID="9396b799-3109-413a-a21f-0df0b62ef0c5" containerName="manager" containerID="cri-o://551305b4b9dd9b15f1047367fa7599259d651f13ef99a118bb94b8b9d214005e" gracePeriod=2 Jul 07 00:06:09.841192 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:09.841146 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" podStartSLOduration=0.852975929 podStartE2EDuration="4.841128227s" totalImagesPullingTime="3.988152298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:09.840225135 +0000 UTC m=+535.812709797" watchObservedRunningTime="2026-07-07 00:06:09.841128227 +0000 UTC m=+535.813612887" Jul 07 00:06:10.069019 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.068988 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:10.071292 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:06:10.071254 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-jxphr\" is forbidden: User \"system:node:ip-10-0-130-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-52.ec2.internal' and this object" podUID="9396b799-3109-413a-a21f-0df0b62ef0c5" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:10.151550 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.151524 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxr6w\" (UniqueName: \"kubernetes.io/projected/9396b799-3109-413a-a21f-0df0b62ef0c5-kube-api-access-cxr6w\") pod \"9396b799-3109-413a-a21f-0df0b62ef0c5\" (UID: \"9396b799-3109-413a-a21f-0df0b62ef0c5\") " Jul 07 00:06:10.151708 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.151568 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9396b799-3109-413a-a21f-0df0b62ef0c5-extensions-socket-volume\") pod \"9396b799-3109-413a-a21f-0df0b62ef0c5\" (UID: \"9396b799-3109-413a-a21f-0df0b62ef0c5\") " Jul 07 00:06:10.151875 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.151847 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9396b799-3109-413a-a21f-0df0b62ef0c5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9396b799-3109-413a-a21f-0df0b62ef0c5" (UID: "9396b799-3109-413a-a21f-0df0b62ef0c5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:06:10.153674 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.153643 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9396b799-3109-413a-a21f-0df0b62ef0c5-kube-api-access-cxr6w" (OuterVolumeSpecName: "kube-api-access-cxr6w") pod "9396b799-3109-413a-a21f-0df0b62ef0c5" (UID: "9396b799-3109-413a-a21f-0df0b62ef0c5"). InnerVolumeSpecName "kube-api-access-cxr6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:06:10.252835 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.252778 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxr6w\" (UniqueName: \"kubernetes.io/projected/9396b799-3109-413a-a21f-0df0b62ef0c5-kube-api-access-cxr6w\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:06:10.252835 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.252826 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9396b799-3109-413a-a21f-0df0b62ef0c5-extensions-socket-volume\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:06:10.583110 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.583018 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9396b799-3109-413a-a21f-0df0b62ef0c5" path="/var/lib/kubelet/pods/9396b799-3109-413a-a21f-0df0b62ef0c5/volumes" Jul 07 00:06:10.828318 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.828285 2562 generic.go:356] "Generic (PLEG): container finished" podID="9396b799-3109-413a-a21f-0df0b62ef0c5" containerID="551305b4b9dd9b15f1047367fa7599259d651f13ef99a118bb94b8b9d214005e" exitCode=2 Jul 07 00:06:10.828779 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.828336 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:10.828779 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.828396 2562 scope.go:117] "RemoveContainer" containerID="551305b4b9dd9b15f1047367fa7599259d651f13ef99a118bb94b8b9d214005e" Jul 07 00:06:10.833457 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:06:10.833384 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-jxphr\" is forbidden: User \"system:node:ip-10-0-130-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-52.ec2.internal' and this object" podUID="9396b799-3109-413a-a21f-0df0b62ef0c5" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jxphr" Jul 07 00:06:10.837728 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.837708 2562 scope.go:117] "RemoveContainer" containerID="551305b4b9dd9b15f1047367fa7599259d651f13ef99a118bb94b8b9d214005e" Jul 07 00:06:10.837992 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:06:10.837972 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"551305b4b9dd9b15f1047367fa7599259d651f13ef99a118bb94b8b9d214005e\": container with ID starting with 551305b4b9dd9b15f1047367fa7599259d651f13ef99a118bb94b8b9d214005e not found: ID does not exist" containerID="551305b4b9dd9b15f1047367fa7599259d651f13ef99a118bb94b8b9d214005e" Jul 07 00:06:10.838047 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:10.838005 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"551305b4b9dd9b15f1047367fa7599259d651f13ef99a118bb94b8b9d214005e"} err="failed to get container status \"551305b4b9dd9b15f1047367fa7599259d651f13ef99a118bb94b8b9d214005e\": rpc error: code = NotFound desc = could not find container \"551305b4b9dd9b15f1047367fa7599259d651f13ef99a118bb94b8b9d214005e\": container with ID starting with 551305b4b9dd9b15f1047367fa7599259d651f13ef99a118bb94b8b9d214005e not found: ID does not exist" Jul 07 00:06:20.831311 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:20.831278 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:33.557901 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.557864 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs"] Jul 07 00:06:33.558403 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.558092 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" podUID="2018ca11-5106-4a27-87e3-0d0f58785864" containerName="manager" containerID="cri-o://179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba" gracePeriod=10 Jul 07 00:06:33.810039 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.809972 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:33.864551 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.864521 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tqsp\" (UniqueName: \"kubernetes.io/projected/2018ca11-5106-4a27-87e3-0d0f58785864-kube-api-access-2tqsp\") pod \"2018ca11-5106-4a27-87e3-0d0f58785864\" (UID: \"2018ca11-5106-4a27-87e3-0d0f58785864\") " Jul 07 00:06:33.864738 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.864583 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2018ca11-5106-4a27-87e3-0d0f58785864-extensions-socket-volume\") pod \"2018ca11-5106-4a27-87e3-0d0f58785864\" (UID: \"2018ca11-5106-4a27-87e3-0d0f58785864\") " Jul 07 00:06:33.865091 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.865043 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2018ca11-5106-4a27-87e3-0d0f58785864-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2018ca11-5106-4a27-87e3-0d0f58785864" (UID: "2018ca11-5106-4a27-87e3-0d0f58785864"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:06:33.866697 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.866672 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2018ca11-5106-4a27-87e3-0d0f58785864-kube-api-access-2tqsp" (OuterVolumeSpecName: "kube-api-access-2tqsp") pod "2018ca11-5106-4a27-87e3-0d0f58785864" (UID: "2018ca11-5106-4a27-87e3-0d0f58785864"). InnerVolumeSpecName "kube-api-access-2tqsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:06:33.938094 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.938031 2562 generic.go:356] "Generic (PLEG): container finished" podID="2018ca11-5106-4a27-87e3-0d0f58785864" containerID="179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba" exitCode=0 Jul 07 00:06:33.938300 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.938110 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" Jul 07 00:06:33.938300 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.938121 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" event={"ID":"2018ca11-5106-4a27-87e3-0d0f58785864","Type":"ContainerDied","Data":"179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba"} Jul 07 00:06:33.938300 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.938194 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs" event={"ID":"2018ca11-5106-4a27-87e3-0d0f58785864","Type":"ContainerDied","Data":"34ca77135deb139d85e23535f2eb079f212d16a83bd2989efcd89ab9f8e68456"} Jul 07 00:06:33.938300 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.938228 2562 scope.go:117] "RemoveContainer" containerID="179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba" Jul 07 00:06:33.947750 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.947723 2562 scope.go:117] "RemoveContainer" containerID="179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba" Jul 07 00:06:33.948026 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:06:33.948002 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba\": container with ID starting with 179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba not found: ID does not exist" containerID="179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba" Jul 07 00:06:33.948145 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.948035 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba"} err="failed to get container status \"179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba\": rpc error: code = NotFound desc = could not find container \"179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba\": container with ID starting with 179a481bfd560464cd7b9858fbefb4c392a6b97007a5858dff5b9122d2e747ba not found: ID does not exist" Jul 07 00:06:33.969738 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.966598 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs"] Jul 07 00:06:33.969738 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.967357 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tqsp\" (UniqueName: \"kubernetes.io/projected/2018ca11-5106-4a27-87e3-0d0f58785864-kube-api-access-2tqsp\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:06:33.969738 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.967383 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2018ca11-5106-4a27-87e3-0d0f58785864-extensions-socket-volume\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:06:33.973038 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:33.973012 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-646gs"] Jul 07 00:06:34.582855 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:34.582817 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2018ca11-5106-4a27-87e3-0d0f58785864" path="/var/lib/kubelet/pods/2018ca11-5106-4a27-87e3-0d0f58785864/volumes" Jul 07 00:06:52.798949 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.798906 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vddg"] Jul 07 00:06:52.799403 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.799223 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2018ca11-5106-4a27-87e3-0d0f58785864" containerName="manager" Jul 07 00:06:52.799403 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.799235 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2018ca11-5106-4a27-87e3-0d0f58785864" containerName="manager" Jul 07 00:06:52.799403 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.799258 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9396b799-3109-413a-a21f-0df0b62ef0c5" containerName="manager" Jul 07 00:06:52.799403 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.799265 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9396b799-3109-413a-a21f-0df0b62ef0c5" containerName="manager" Jul 07 00:06:52.799403 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.799321 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2018ca11-5106-4a27-87e3-0d0f58785864" containerName="manager" Jul 07 00:06:52.799403 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.799328 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9396b799-3109-413a-a21f-0df0b62ef0c5" containerName="manager" Jul 07 00:06:52.801113 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.801096 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:06:52.803870 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.803847 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 00:06:52.804012 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.803909 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 00:06:52.804729 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.804712 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 00:06:52.804729 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.804720 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v659t\"" Jul 07 00:06:52.814287 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.814255 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vddg"] Jul 07 00:06:52.898626 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.898586 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vddg"] Jul 07 00:06:52.935689 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.935655 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2xbl\" (UniqueName: \"kubernetes.io/projected/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-kube-api-access-b2xbl\") pod \"limitador-limitador-64c94fd64f-4vddg\" (UID: \"19be42de-3c58-47e5-904e-f3a8a9f9ad7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:06:52.935867 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:52.935724 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-config-file\") pod \"limitador-limitador-64c94fd64f-4vddg\" (UID: \"19be42de-3c58-47e5-904e-f3a8a9f9ad7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:06:53.036811 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:53.036774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2xbl\" (UniqueName: \"kubernetes.io/projected/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-kube-api-access-b2xbl\") pod \"limitador-limitador-64c94fd64f-4vddg\" (UID: \"19be42de-3c58-47e5-904e-f3a8a9f9ad7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:06:53.037006 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:53.036864 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-config-file\") pod \"limitador-limitador-64c94fd64f-4vddg\" (UID: \"19be42de-3c58-47e5-904e-f3a8a9f9ad7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:06:53.037748 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:53.037720 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-config-file\") pod \"limitador-limitador-64c94fd64f-4vddg\" (UID: \"19be42de-3c58-47e5-904e-f3a8a9f9ad7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:06:53.045361 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:53.045336 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2xbl\" (UniqueName: \"kubernetes.io/projected/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-kube-api-access-b2xbl\") pod \"limitador-limitador-64c94fd64f-4vddg\" (UID: \"19be42de-3c58-47e5-904e-f3a8a9f9ad7b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:06:53.113340 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:53.113308 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:06:53.255288 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:53.255261 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vddg"] Jul 07 00:06:53.257700 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:06:53.257651 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19be42de_3c58_47e5_904e_f3a8a9f9ad7b.slice/crio-629f4ed85b2d9c10c13341c8947a55c4c9e71ed2c696995a9892282211d45b5e WatchSource:0}: Error finding container 629f4ed85b2d9c10c13341c8947a55c4c9e71ed2c696995a9892282211d45b5e: Status 404 returned error can't find the container with id 629f4ed85b2d9c10c13341c8947a55c4c9e71ed2c696995a9892282211d45b5e Jul 07 00:06:54.040642 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:54.040509 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" event={"ID":"19be42de-3c58-47e5-904e-f3a8a9f9ad7b","Type":"ContainerStarted","Data":"629f4ed85b2d9c10c13341c8947a55c4c9e71ed2c696995a9892282211d45b5e"} Jul 07 00:06:57.060898 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:57.060857 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" event={"ID":"19be42de-3c58-47e5-904e-f3a8a9f9ad7b","Type":"ContainerStarted","Data":"3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549"} Jul 07 00:06:57.061372 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:57.060941 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:06:57.076298 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:06:57.076232 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" podStartSLOduration=2.3372960799999998 podStartE2EDuration="5.076212239s" totalImagesPullingTime="2.738916159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:57.075908944 +0000 UTC m=+583.048393608" watchObservedRunningTime="2026-07-07 00:06:57.076212239 +0000 UTC m=+583.048696900" Jul 07 00:07:07.446311 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:07.446270 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vddg"] Jul 07 00:07:07.446878 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:07.446494 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" podUID="19be42de-3c58-47e5-904e-f3a8a9f9ad7b" containerName="limitador" containerID="cri-o://3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549" gracePeriod=30 Jul 07 00:07:07.447438 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:07.447246 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:07:07.989139 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:07.989111 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:07:08.062715 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.062613 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-config-file\") pod \"19be42de-3c58-47e5-904e-f3a8a9f9ad7b\" (UID: \"19be42de-3c58-47e5-904e-f3a8a9f9ad7b\") " Jul 07 00:07:08.062892 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.062744 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2xbl\" (UniqueName: \"kubernetes.io/projected/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-kube-api-access-b2xbl\") pod \"19be42de-3c58-47e5-904e-f3a8a9f9ad7b\" (UID: \"19be42de-3c58-47e5-904e-f3a8a9f9ad7b\") " Jul 07 00:07:08.062984 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.062959 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-config-file" (OuterVolumeSpecName: "config-file") pod "19be42de-3c58-47e5-904e-f3a8a9f9ad7b" (UID: "19be42de-3c58-47e5-904e-f3a8a9f9ad7b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:07:08.064869 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.064840 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-kube-api-access-b2xbl" (OuterVolumeSpecName: "kube-api-access-b2xbl") pod "19be42de-3c58-47e5-904e-f3a8a9f9ad7b" (UID: "19be42de-3c58-47e5-904e-f3a8a9f9ad7b"). InnerVolumeSpecName "kube-api-access-b2xbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:07:08.112376 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.112342 2562 generic.go:356] "Generic (PLEG): container finished" podID="19be42de-3c58-47e5-904e-f3a8a9f9ad7b" containerID="3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549" exitCode=0 Jul 07 00:07:08.112543 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.112382 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" Jul 07 00:07:08.112543 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.112420 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" event={"ID":"19be42de-3c58-47e5-904e-f3a8a9f9ad7b","Type":"ContainerDied","Data":"3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549"} Jul 07 00:07:08.112543 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.112485 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vddg" event={"ID":"19be42de-3c58-47e5-904e-f3a8a9f9ad7b","Type":"ContainerDied","Data":"629f4ed85b2d9c10c13341c8947a55c4c9e71ed2c696995a9892282211d45b5e"} Jul 07 00:07:08.112543 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.112518 2562 scope.go:117] "RemoveContainer" containerID="3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549" Jul 07 00:07:08.122145 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.122124 2562 scope.go:117] "RemoveContainer" containerID="3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549" Jul 07 00:07:08.122436 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:07:08.122419 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549\": container with ID starting with 3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549 not found: ID does not exist" containerID="3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549" Jul 07 00:07:08.122484 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.122444 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549"} err="failed to get container status \"3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549\": rpc error: code = NotFound desc = could not find container \"3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549\": container with ID starting with 3674c1c6b86e9fa62358ea722d44331ee6a46e004fec81f141a48f9f61d5e549 not found: ID does not exist" Jul 07 00:07:08.137378 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.137347 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vddg"] Jul 07 00:07:08.143149 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.143118 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vddg"] Jul 07 00:07:08.163536 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.163505 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2xbl\" (UniqueName: \"kubernetes.io/projected/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-kube-api-access-b2xbl\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:07:08.163536 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.163536 2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/19be42de-3c58-47e5-904e-f3a8a9f9ad7b-config-file\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:07:08.582909 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:08.582869 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19be42de-3c58-47e5-904e-f3a8a9f9ad7b" path="/var/lib/kubelet/pods/19be42de-3c58-47e5-904e-f3a8a9f9ad7b/volumes" Jul 07 00:07:11.682270 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.682190 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-95hzk"] Jul 07 00:07:11.682786 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.682764 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19be42de-3c58-47e5-904e-f3a8a9f9ad7b" containerName="limitador" Jul 07 00:07:11.682858 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.682788 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="19be42de-3c58-47e5-904e-f3a8a9f9ad7b" containerName="limitador" Jul 07 00:07:11.682903 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.682877 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="19be42de-3c58-47e5-904e-f3a8a9f9ad7b" containerName="limitador" Jul 07 00:07:11.685535 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.685517 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" Jul 07 00:07:11.688494 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.688472 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-g4fvq\"" Jul 07 00:07:11.688494 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.688485 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 07 00:07:11.689353 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.689333 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 07 00:07:11.689449 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.689389 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 07 00:07:11.690735 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.690712 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptc92\" (UniqueName: \"kubernetes.io/projected/dda30698-4409-4371-851d-cd952e5e29d5-kube-api-access-ptc92\") pod \"postgres-69fb4594cf-95hzk\" (UID: \"dda30698-4409-4371-851d-cd952e5e29d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" Jul 07 00:07:11.690916 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.690895 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dda30698-4409-4371-851d-cd952e5e29d5-data\") pod \"postgres-69fb4594cf-95hzk\" (UID: \"dda30698-4409-4371-851d-cd952e5e29d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" Jul 07 00:07:11.693733 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.693714 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-95hzk"] Jul 07 00:07:11.792352 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.792301 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptc92\" (UniqueName: \"kubernetes.io/projected/dda30698-4409-4371-851d-cd952e5e29d5-kube-api-access-ptc92\") pod \"postgres-69fb4594cf-95hzk\" (UID: \"dda30698-4409-4371-851d-cd952e5e29d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" Jul 07 00:07:11.792548 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.792401 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dda30698-4409-4371-851d-cd952e5e29d5-data\") pod \"postgres-69fb4594cf-95hzk\" (UID: \"dda30698-4409-4371-851d-cd952e5e29d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" Jul 07 00:07:11.792937 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.792902 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dda30698-4409-4371-851d-cd952e5e29d5-data\") pod \"postgres-69fb4594cf-95hzk\" (UID: \"dda30698-4409-4371-851d-cd952e5e29d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" Jul 07 00:07:11.800081 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.800045 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptc92\" (UniqueName: \"kubernetes.io/projected/dda30698-4409-4371-851d-cd952e5e29d5-kube-api-access-ptc92\") pod \"postgres-69fb4594cf-95hzk\" (UID: \"dda30698-4409-4371-851d-cd952e5e29d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" Jul 07 00:07:11.996539 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:11.996448 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" Jul 07 00:07:12.131054 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:12.131031 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-95hzk"] Jul 07 00:07:12.133257 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:07:12.133215 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda30698_4409_4371_851d_cd952e5e29d5.slice/crio-0eb9a737fcddfb3dd06b65426f41dfb933de10b098570b8d337963bb54301305 WatchSource:0}: Error finding container 0eb9a737fcddfb3dd06b65426f41dfb933de10b098570b8d337963bb54301305: Status 404 returned error can't find the container with id 0eb9a737fcddfb3dd06b65426f41dfb933de10b098570b8d337963bb54301305 Jul 07 00:07:13.144033 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:13.143966 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" event={"ID":"dda30698-4409-4371-851d-cd952e5e29d5","Type":"ContainerStarted","Data":"0eb9a737fcddfb3dd06b65426f41dfb933de10b098570b8d337963bb54301305"} Jul 07 00:07:16.648258 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:16.648231 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 07 00:07:17.166589 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:17.166552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" event={"ID":"dda30698-4409-4371-851d-cd952e5e29d5","Type":"ContainerStarted","Data":"d5dc88eb90ebfaab94679867f8b5f504ef399a36ca15d5b8c1453fe254960b9a"} Jul 07 00:07:17.166779 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:17.166654 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" Jul 07 00:07:17.182763 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:17.182711 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" podStartSLOduration=1.671784922 podStartE2EDuration="6.182693298s" totalImagesPullingTime="4.510908376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:07:17.181832852 +0000 UTC m=+603.154317514" watchObservedRunningTime="2026-07-07 00:07:17.182693298 +0000 UTC m=+603.155177958" Jul 07 00:07:23.201280 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:23.201252 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-95hzk" Jul 07 00:07:30.306579 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.306547 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86b94f68c-2m6kd"] Jul 07 00:07:30.331728 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.331683 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86b94f68c-2m6kd"] Jul 07 00:07:30.331913 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.331818 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:30.334640 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.334618 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 00:07:30.334921 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.334901 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-thk6q\"" Jul 07 00:07:30.335164 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.335150 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 00:07:30.457028 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.456993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-webhook-cert\") pod \"maas-controller-86b94f68c-2m6kd\" (UID: \"3a4cf4a6-a737-41be-9ef6-e5a771c8b949\") " pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:30.457242 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.457038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw8vz\" (UniqueName: \"kubernetes.io/projected/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-kube-api-access-qw8vz\") pod \"maas-controller-86b94f68c-2m6kd\" (UID: \"3a4cf4a6-a737-41be-9ef6-e5a771c8b949\") " pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:30.478893 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.478854 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5868db4bd6-8lxdw"] Jul 07 00:07:30.484267 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.484241 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:07:30.488521 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.488491 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5868db4bd6-8lxdw"] Jul 07 00:07:30.557903 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.557808 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-webhook-cert\") pod \"maas-controller-86b94f68c-2m6kd\" (UID: \"3a4cf4a6-a737-41be-9ef6-e5a771c8b949\") " pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:30.557903 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.557859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhddg\" (UniqueName: \"kubernetes.io/projected/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-kube-api-access-jhddg\") pod \"maas-controller-5868db4bd6-8lxdw\" (UID: \"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8\") " pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:07:30.557903 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.557892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw8vz\" (UniqueName: \"kubernetes.io/projected/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-kube-api-access-qw8vz\") pod \"maas-controller-86b94f68c-2m6kd\" (UID: \"3a4cf4a6-a737-41be-9ef6-e5a771c8b949\") " pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:30.558152 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.557937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-webhook-cert\") pod \"maas-controller-5868db4bd6-8lxdw\" (UID: \"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8\") " pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:07:30.560448 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.560427 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-webhook-cert\") pod \"maas-controller-86b94f68c-2m6kd\" (UID: \"3a4cf4a6-a737-41be-9ef6-e5a771c8b949\") " pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:30.567143 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.567113 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw8vz\" (UniqueName: \"kubernetes.io/projected/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-kube-api-access-qw8vz\") pod \"maas-controller-86b94f68c-2m6kd\" (UID: \"3a4cf4a6-a737-41be-9ef6-e5a771c8b949\") " pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:30.643117 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.643087 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:30.659379 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.659351 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-webhook-cert\") pod \"maas-controller-5868db4bd6-8lxdw\" (UID: \"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8\") " pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:07:30.659513 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.659446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhddg\" (UniqueName: \"kubernetes.io/projected/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-kube-api-access-jhddg\") pod \"maas-controller-5868db4bd6-8lxdw\" (UID: \"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8\") " pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:07:30.661842 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.661818 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-webhook-cert\") pod \"maas-controller-5868db4bd6-8lxdw\" (UID: \"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8\") " pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:07:30.668795 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.668768 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhddg\" (UniqueName: \"kubernetes.io/projected/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-kube-api-access-jhddg\") pod \"maas-controller-5868db4bd6-8lxdw\" (UID: \"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8\") " pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:07:30.775450 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.775423 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86b94f68c-2m6kd"] Jul 07 00:07:30.777114 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:07:30.777087 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4cf4a6_a737_41be_9ef6_e5a771c8b949.slice/crio-76a9835903f66fa257bcb3bef42b11f6c9969cfcd8ec7807a6b272464ea15837 WatchSource:0}: Error finding container 76a9835903f66fa257bcb3bef42b11f6c9969cfcd8ec7807a6b272464ea15837: Status 404 returned error can't find the container with id 76a9835903f66fa257bcb3bef42b11f6c9969cfcd8ec7807a6b272464ea15837 Jul 07 00:07:30.778516 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.778500 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 00:07:30.795978 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.795947 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:07:30.925273 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:30.925248 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5868db4bd6-8lxdw"] Jul 07 00:07:30.927154 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:07:30.927126 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6dc75d2_5a01_4af1_8b23_69cd3aa4a0b8.slice/crio-267464da1ca8ffe2c2b9d516f72ee9b73797b7e1f4bf0976e96b684a3b5bceed WatchSource:0}: Error finding container 267464da1ca8ffe2c2b9d516f72ee9b73797b7e1f4bf0976e96b684a3b5bceed: Status 404 returned error can't find the container with id 267464da1ca8ffe2c2b9d516f72ee9b73797b7e1f4bf0976e96b684a3b5bceed Jul 07 00:07:31.230960 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:31.230922 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" event={"ID":"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8","Type":"ContainerStarted","Data":"267464da1ca8ffe2c2b9d516f72ee9b73797b7e1f4bf0976e96b684a3b5bceed"} Jul 07 00:07:31.232022 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:31.231993 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86b94f68c-2m6kd" event={"ID":"3a4cf4a6-a737-41be-9ef6-e5a771c8b949","Type":"ContainerStarted","Data":"76a9835903f66fa257bcb3bef42b11f6c9969cfcd8ec7807a6b272464ea15837"} Jul 07 00:07:33.246262 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:33.246167 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" event={"ID":"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8","Type":"ContainerStarted","Data":"c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff"} Jul 07 00:07:33.246671 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:33.246265 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:07:33.247763 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:33.247737 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86b94f68c-2m6kd" event={"ID":"3a4cf4a6-a737-41be-9ef6-e5a771c8b949","Type":"ContainerStarted","Data":"14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c"} Jul 07 00:07:33.247882 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:33.247787 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:33.278970 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:33.278911 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" podStartSLOduration=1.202463922 podStartE2EDuration="3.278892264s" totalImagesPullingTime="2.076428342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:07:33.276950311 +0000 UTC m=+619.249434986" watchObservedRunningTime="2026-07-07 00:07:33.278892264 +0000 UTC m=+619.251376926" Jul 07 00:07:33.324604 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:33.324540 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86b94f68c-2m6kd" podStartSLOduration=1.108918929 podStartE2EDuration="3.324518855s" totalImagesPullingTime="2.215599926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:07:33.322918262 +0000 UTC m=+619.295402924" watchObservedRunningTime="2026-07-07 00:07:33.324518855 +0000 UTC m=+619.297003516" Jul 07 00:07:44.258873 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:44.258793 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:07:44.258873 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:44.258862 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:44.320356 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:44.320318 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86b94f68c-2m6kd"] Jul 07 00:07:44.320629 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:44.320576 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-86b94f68c-2m6kd" podUID="3a4cf4a6-a737-41be-9ef6-e5a771c8b949" containerName="manager" containerID="cri-o://14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c" gracePeriod=30 Jul 07 00:07:44.554362 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:44.554339 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:44.683179 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:44.683149 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw8vz\" (UniqueName: \"kubernetes.io/projected/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-kube-api-access-qw8vz\") pod \"3a4cf4a6-a737-41be-9ef6-e5a771c8b949\" (UID: \"3a4cf4a6-a737-41be-9ef6-e5a771c8b949\") " Jul 07 00:07:44.683345 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:44.683194 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-webhook-cert\") pod \"3a4cf4a6-a737-41be-9ef6-e5a771c8b949\" (UID: \"3a4cf4a6-a737-41be-9ef6-e5a771c8b949\") " Jul 07 00:07:44.685358 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:44.685332 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-kube-api-access-qw8vz" (OuterVolumeSpecName: "kube-api-access-qw8vz") pod "3a4cf4a6-a737-41be-9ef6-e5a771c8b949" (UID: "3a4cf4a6-a737-41be-9ef6-e5a771c8b949"). InnerVolumeSpecName "kube-api-access-qw8vz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:07:44.685471 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:44.685352 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3a4cf4a6-a737-41be-9ef6-e5a771c8b949" (UID: "3a4cf4a6-a737-41be-9ef6-e5a771c8b949"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:07:44.784411 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:44.784367 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw8vz\" (UniqueName: \"kubernetes.io/projected/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-kube-api-access-qw8vz\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:07:44.784411 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:44.784407 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a4cf4a6-a737-41be-9ef6-e5a771c8b949-webhook-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:07:45.304338 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:45.304299 2562 generic.go:356] "Generic (PLEG): container finished" podID="3a4cf4a6-a737-41be-9ef6-e5a771c8b949" containerID="14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c" exitCode=0 Jul 07 00:07:45.304833 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:45.304352 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86b94f68c-2m6kd" Jul 07 00:07:45.304833 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:45.304391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86b94f68c-2m6kd" event={"ID":"3a4cf4a6-a737-41be-9ef6-e5a771c8b949","Type":"ContainerDied","Data":"14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c"} Jul 07 00:07:45.304833 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:45.304439 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86b94f68c-2m6kd" event={"ID":"3a4cf4a6-a737-41be-9ef6-e5a771c8b949","Type":"ContainerDied","Data":"76a9835903f66fa257bcb3bef42b11f6c9969cfcd8ec7807a6b272464ea15837"} Jul 07 00:07:45.304833 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:45.304461 2562 scope.go:117] "RemoveContainer" containerID="14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c" Jul 07 00:07:45.313472 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:45.313454 2562 scope.go:117] "RemoveContainer" containerID="14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c" Jul 07 00:07:45.313748 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:07:45.313730 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c\": container with ID starting with 14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c not found: ID does not exist" containerID="14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c" Jul 07 00:07:45.313803 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:45.313756 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c"} err="failed to get container status \"14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c\": rpc error: code = NotFound desc = could not find container \"14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c\": container with ID starting with 14e899bfcb6da5f54900e62e5c3df0c9f99340575e662334d415fb64f1fb1c9c not found: ID does not exist" Jul 07 00:07:45.328842 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:45.328820 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86b94f68c-2m6kd"] Jul 07 00:07:45.334821 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:45.334800 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-86b94f68c-2m6kd"] Jul 07 00:07:46.582441 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:07:46.582411 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a4cf4a6-a737-41be-9ef6-e5a771c8b949" path="/var/lib/kubelet/pods/3a4cf4a6-a737-41be-9ef6-e5a771c8b949/volumes" Jul 07 00:08:02.019806 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.019764 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9"] Jul 07 00:08:02.020306 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.020272 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a4cf4a6-a737-41be-9ef6-e5a771c8b949" containerName="manager" Jul 07 00:08:02.020306 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.020287 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a4cf4a6-a737-41be-9ef6-e5a771c8b949" containerName="manager" Jul 07 00:08:02.020386 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.020370 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a4cf4a6-a737-41be-9ef6-e5a771c8b949" containerName="manager" Jul 07 00:08:02.024875 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.024854 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:02.027434 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.027412 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 07 00:08:02.027522 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.027463 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lq9jt\"" Jul 07 00:08:02.029552 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.029531 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlrnx\" (UniqueName: \"kubernetes.io/projected/9d79020d-f11b-4e3d-9007-278c59364ded-kube-api-access-tlrnx\") pod \"maas-api-cc7b7bfdb-8gkj9\" (UID: \"9d79020d-f11b-4e3d-9007-278c59364ded\") " pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:02.029661 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.029568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d79020d-f11b-4e3d-9007-278c59364ded-maas-api-tls\") pod \"maas-api-cc7b7bfdb-8gkj9\" (UID: \"9d79020d-f11b-4e3d-9007-278c59364ded\") " pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:02.029661 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.029588 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d79020d-f11b-4e3d-9007-278c59364ded-openshift-service-ca\") pod \"maas-api-cc7b7bfdb-8gkj9\" (UID: \"9d79020d-f11b-4e3d-9007-278c59364ded\") " pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:02.034496 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.034472 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9"] Jul 07 00:08:02.130390 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.130347 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlrnx\" (UniqueName: \"kubernetes.io/projected/9d79020d-f11b-4e3d-9007-278c59364ded-kube-api-access-tlrnx\") pod \"maas-api-cc7b7bfdb-8gkj9\" (UID: \"9d79020d-f11b-4e3d-9007-278c59364ded\") " pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:02.130390 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.130392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d79020d-f11b-4e3d-9007-278c59364ded-maas-api-tls\") pod \"maas-api-cc7b7bfdb-8gkj9\" (UID: \"9d79020d-f11b-4e3d-9007-278c59364ded\") " pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:02.130653 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.130413 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d79020d-f11b-4e3d-9007-278c59364ded-openshift-service-ca\") pod \"maas-api-cc7b7bfdb-8gkj9\" (UID: \"9d79020d-f11b-4e3d-9007-278c59364ded\") " pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:02.130653 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:08:02.130518 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 07 00:08:02.130653 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:08:02.130591 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d79020d-f11b-4e3d-9007-278c59364ded-maas-api-tls podName:9d79020d-f11b-4e3d-9007-278c59364ded nodeName:}" failed. No retries permitted until 2026-07-07 00:08:02.630574199 +0000 UTC m=+648.603058848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9d79020d-f11b-4e3d-9007-278c59364ded-maas-api-tls") pod "maas-api-cc7b7bfdb-8gkj9" (UID: "9d79020d-f11b-4e3d-9007-278c59364ded") : secret "maas-api-serving-cert" not found Jul 07 00:08:02.131237 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.131211 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d79020d-f11b-4e3d-9007-278c59364ded-openshift-service-ca\") pod \"maas-api-cc7b7bfdb-8gkj9\" (UID: \"9d79020d-f11b-4e3d-9007-278c59364ded\") " pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:02.141871 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.141838 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlrnx\" (UniqueName: \"kubernetes.io/projected/9d79020d-f11b-4e3d-9007-278c59364ded-kube-api-access-tlrnx\") pod \"maas-api-cc7b7bfdb-8gkj9\" (UID: \"9d79020d-f11b-4e3d-9007-278c59364ded\") " pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:02.499929 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.499890 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-75qqk"] Jul 07 00:08:02.503512 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.503493 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" Jul 07 00:08:02.506124 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.506100 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 00:08:02.506246 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.506100 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-gsq2s\"" Jul 07 00:08:02.517482 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.517452 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-75qqk"] Jul 07 00:08:02.528382 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.528352 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-svzf4"] Jul 07 00:08:02.532154 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.532133 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-svzf4" Jul 07 00:08:02.533498 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.533474 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxdpm\" (UniqueName: \"kubernetes.io/projected/96a24813-99cc-493c-84d1-5c9872de996a-kube-api-access-bxdpm\") pod \"payload-pre-processing-7964457696-75qqk\" (UID: \"96a24813-99cc-493c-84d1-5c9872de996a\") " pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" Jul 07 00:08:02.533716 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.533696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96a24813-99cc-493c-84d1-5c9872de996a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-75qqk\" (UID: \"96a24813-99cc-493c-84d1-5c9872de996a\") " pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" Jul 07 00:08:02.542612 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.542591 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-svzf4"] Jul 07 00:08:02.634676 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.634644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxdpm\" (UniqueName: \"kubernetes.io/projected/96a24813-99cc-493c-84d1-5c9872de996a-kube-api-access-bxdpm\") pod \"payload-pre-processing-7964457696-75qqk\" (UID: \"96a24813-99cc-493c-84d1-5c9872de996a\") " pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" Jul 07 00:08:02.634881 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.634698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d79020d-f11b-4e3d-9007-278c59364ded-maas-api-tls\") pod \"maas-api-cc7b7bfdb-8gkj9\" (UID: \"9d79020d-f11b-4e3d-9007-278c59364ded\") " pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:02.634881 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.634748 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96a24813-99cc-493c-84d1-5c9872de996a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-75qqk\" (UID: \"96a24813-99cc-493c-84d1-5c9872de996a\") " pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" Jul 07 00:08:02.634881 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.634787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54a92b1d-f008-412d-a789-da0f3bad7dd7-plugins-config-volume\") pod \"payload-processing-58f86c7798-svzf4\" (UID: \"54a92b1d-f008-412d-a789-da0f3bad7dd7\") " pod="openshift-ingress/payload-processing-58f86c7798-svzf4" Jul 07 00:08:02.634881 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.634832 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkd9s\" (UniqueName: \"kubernetes.io/projected/54a92b1d-f008-412d-a789-da0f3bad7dd7-kube-api-access-dkd9s\") pod \"payload-processing-58f86c7798-svzf4\" (UID: \"54a92b1d-f008-412d-a789-da0f3bad7dd7\") " pod="openshift-ingress/payload-processing-58f86c7798-svzf4" Jul 07 00:08:02.634881 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:08:02.634870 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 07 00:08:02.635159 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:08:02.634942 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d79020d-f11b-4e3d-9007-278c59364ded-maas-api-tls podName:9d79020d-f11b-4e3d-9007-278c59364ded nodeName:}" failed. No retries permitted until 2026-07-07 00:08:03.634921641 +0000 UTC m=+649.607406300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9d79020d-f11b-4e3d-9007-278c59364ded-maas-api-tls") pod "maas-api-cc7b7bfdb-8gkj9" (UID: "9d79020d-f11b-4e3d-9007-278c59364ded") : secret "maas-api-serving-cert" not found Jul 07 00:08:02.635490 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.635471 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96a24813-99cc-493c-84d1-5c9872de996a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-75qqk\" (UID: \"96a24813-99cc-493c-84d1-5c9872de996a\") " pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" Jul 07 00:08:02.646611 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.646581 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxdpm\" (UniqueName: \"kubernetes.io/projected/96a24813-99cc-493c-84d1-5c9872de996a-kube-api-access-bxdpm\") pod \"payload-pre-processing-7964457696-75qqk\" (UID: \"96a24813-99cc-493c-84d1-5c9872de996a\") " pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" Jul 07 00:08:02.735524 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.735485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54a92b1d-f008-412d-a789-da0f3bad7dd7-plugins-config-volume\") pod \"payload-processing-58f86c7798-svzf4\" (UID: \"54a92b1d-f008-412d-a789-da0f3bad7dd7\") " pod="openshift-ingress/payload-processing-58f86c7798-svzf4" Jul 07 00:08:02.735524 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.735531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkd9s\" (UniqueName: \"kubernetes.io/projected/54a92b1d-f008-412d-a789-da0f3bad7dd7-kube-api-access-dkd9s\") pod \"payload-processing-58f86c7798-svzf4\" (UID: \"54a92b1d-f008-412d-a789-da0f3bad7dd7\") " pod="openshift-ingress/payload-processing-58f86c7798-svzf4" Jul 07 00:08:02.736363 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.736337 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54a92b1d-f008-412d-a789-da0f3bad7dd7-plugins-config-volume\") pod \"payload-processing-58f86c7798-svzf4\" (UID: \"54a92b1d-f008-412d-a789-da0f3bad7dd7\") " pod="openshift-ingress/payload-processing-58f86c7798-svzf4" Jul 07 00:08:02.745154 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.745129 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkd9s\" (UniqueName: \"kubernetes.io/projected/54a92b1d-f008-412d-a789-da0f3bad7dd7-kube-api-access-dkd9s\") pod \"payload-processing-58f86c7798-svzf4\" (UID: \"54a92b1d-f008-412d-a789-da0f3bad7dd7\") " pod="openshift-ingress/payload-processing-58f86c7798-svzf4" Jul 07 00:08:02.816745 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.816651 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" Jul 07 00:08:02.844303 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.844269 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-svzf4" Jul 07 00:08:02.971834 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.971807 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-75qqk"] Jul 07 00:08:02.974475 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:08:02.974445 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a24813_99cc_493c_84d1_5c9872de996a.slice/crio-532a280e4f341072845aaf3f8154ae269f03ec50d4e461a0f382f29899ebae65 WatchSource:0}: Error finding container 532a280e4f341072845aaf3f8154ae269f03ec50d4e461a0f382f29899ebae65: Status 404 returned error can't find the container with id 532a280e4f341072845aaf3f8154ae269f03ec50d4e461a0f382f29899ebae65 Jul 07 00:08:02.994909 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:02.994886 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-svzf4"] Jul 07 00:08:02.996430 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:08:02.996407 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a92b1d_f008_412d_a789_da0f3bad7dd7.slice/crio-1c2ab4a9a3a07bc879d855556ba63c17504e17689dd11b7813901c2251729a18 WatchSource:0}: Error finding container 1c2ab4a9a3a07bc879d855556ba63c17504e17689dd11b7813901c2251729a18: Status 404 returned error can't find the container with id 1c2ab4a9a3a07bc879d855556ba63c17504e17689dd11b7813901c2251729a18 Jul 07 00:08:03.392151 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:03.392116 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-svzf4" event={"ID":"54a92b1d-f008-412d-a789-da0f3bad7dd7","Type":"ContainerStarted","Data":"1c2ab4a9a3a07bc879d855556ba63c17504e17689dd11b7813901c2251729a18"} Jul 07 00:08:03.393178 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:03.393154 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" event={"ID":"96a24813-99cc-493c-84d1-5c9872de996a","Type":"ContainerStarted","Data":"532a280e4f341072845aaf3f8154ae269f03ec50d4e461a0f382f29899ebae65"} Jul 07 00:08:03.645528 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:03.645287 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d79020d-f11b-4e3d-9007-278c59364ded-maas-api-tls\") pod \"maas-api-cc7b7bfdb-8gkj9\" (UID: \"9d79020d-f11b-4e3d-9007-278c59364ded\") " pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:03.648958 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:03.648929 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d79020d-f11b-4e3d-9007-278c59364ded-maas-api-tls\") pod \"maas-api-cc7b7bfdb-8gkj9\" (UID: \"9d79020d-f11b-4e3d-9007-278c59364ded\") " pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:03.837881 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:03.837837 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:04.003720 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:04.003695 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9"] Jul 07 00:08:04.006381 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:08:04.006337 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d79020d_f11b_4e3d_9007_278c59364ded.slice/crio-21e19d83d1262dec1d1d3b468a56c29917b508d5cbcddda9fc2d8625184f82de WatchSource:0}: Error finding container 21e19d83d1262dec1d1d3b468a56c29917b508d5cbcddda9fc2d8625184f82de: Status 404 returned error can't find the container with id 21e19d83d1262dec1d1d3b468a56c29917b508d5cbcddda9fc2d8625184f82de Jul 07 00:08:04.398663 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:04.398625 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" event={"ID":"9d79020d-f11b-4e3d-9007-278c59364ded","Type":"ContainerStarted","Data":"21e19d83d1262dec1d1d3b468a56c29917b508d5cbcddda9fc2d8625184f82de"} Jul 07 00:08:05.406406 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:05.406364 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" event={"ID":"96a24813-99cc-493c-84d1-5c9872de996a","Type":"ContainerStarted","Data":"b2c5e83f839b94d5c8ffda35f12a532fbecb35b0bb9dc8dc948e371bb33f8b05"} Jul 07 00:08:05.406881 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:05.406538 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" Jul 07 00:08:05.407981 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:05.407957 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-svzf4" event={"ID":"54a92b1d-f008-412d-a789-da0f3bad7dd7","Type":"ContainerStarted","Data":"8ffa850130ccda4f73ad1a26d240a2f399d29240f26d1fe52a2c9723c52346fa"} Jul 07 00:08:05.408129 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:05.408097 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-svzf4" Jul 07 00:08:05.423522 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:05.423476 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" podStartSLOduration=1.9777191109999999 podStartE2EDuration="3.423459992s" totalImagesPullingTime="1.445740881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:08:05.421452076 +0000 UTC m=+651.393936738" watchObservedRunningTime="2026-07-07 00:08:05.423459992 +0000 UTC m=+651.395944653" Jul 07 00:08:05.437742 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:05.437017 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-svzf4" podStartSLOduration=2.010596989 podStartE2EDuration="3.436996269s" totalImagesPullingTime="1.42639928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:08:05.436877977 +0000 UTC m=+651.409362639" watchObservedRunningTime="2026-07-07 00:08:05.436996269 +0000 UTC m=+651.409480933" Jul 07 00:08:06.415830 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:06.415789 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" event={"ID":"9d79020d-f11b-4e3d-9007-278c59364ded","Type":"ContainerStarted","Data":"989bb39437cc79a43492e5df5779e84cfbc3b40abeabc90087cb196e40ea9a69"} Jul 07 00:08:06.436396 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:06.436347 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" podStartSLOduration=3.588585198 podStartE2EDuration="5.436329773s" totalImagesPullingTime="1.847744575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:08:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:08:06.435426848 +0000 UTC m=+652.407911510" watchObservedRunningTime="2026-07-07 00:08:06.436329773 +0000 UTC m=+652.408814433" Jul 07 00:08:07.421316 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:07.421286 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:13.431311 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:13.431279 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-cc7b7bfdb-8gkj9" Jul 07 00:08:16.418299 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:16.418265 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-svzf4" Jul 07 00:08:16.418763 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:16.418340 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-75qqk" Jul 07 00:08:30.382618 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.382586 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7"] Jul 07 00:08:30.387312 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.387293 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.391490 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.391459 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 00:08:30.391490 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.391464 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kvp5g\"" Jul 07 00:08:30.391693 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.391518 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 00:08:30.391693 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.391543 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 00:08:30.400036 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.400011 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7"] Jul 07 00:08:30.498466 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.498428 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.498466 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.498471 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rxnn\" (UniqueName: \"kubernetes.io/projected/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-kube-api-access-4rxnn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.498711 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.498496 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.498711 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.498591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.498711 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.498634 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.498711 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.498706 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.599723 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.599679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rxnn\" (UniqueName: \"kubernetes.io/projected/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-kube-api-access-4rxnn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.599939 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.599748 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.599939 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.599801 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.599939 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.599824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.600195 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.599970 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.600195 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.600030 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.600304 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.600206 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.600304 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.600271 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.600402 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.600313 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.602345 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.602314 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.602712 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.602689 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.607733 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.607700 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rxnn\" (UniqueName: \"kubernetes.io/projected/9edd6546-f7a3-47d3-b7c9-bc3cea61599b-kube-api-access-4rxnn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7\" (UID: \"9edd6546-f7a3-47d3-b7c9-bc3cea61599b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.698981 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.698883 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:30.833014 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:30.832988 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7"] Jul 07 00:08:30.835427 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:08:30.835396 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9edd6546_f7a3_47d3_b7c9_bc3cea61599b.slice/crio-5785ec579f958d511d78425dd0652a357e703258f42d7945afeb2daa59cd7b1e WatchSource:0}: Error finding container 5785ec579f958d511d78425dd0652a357e703258f42d7945afeb2daa59cd7b1e: Status 404 returned error can't find the container with id 5785ec579f958d511d78425dd0652a357e703258f42d7945afeb2daa59cd7b1e Jul 07 00:08:31.541661 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:31.541614 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" event={"ID":"9edd6546-f7a3-47d3-b7c9-bc3cea61599b","Type":"ContainerStarted","Data":"5785ec579f958d511d78425dd0652a357e703258f42d7945afeb2daa59cd7b1e"} Jul 07 00:08:32.548302 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:32.548263 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" event={"ID":"9edd6546-f7a3-47d3-b7c9-bc3cea61599b","Type":"ContainerStarted","Data":"e108361a4b766ea901d3e937bed6325e149c0990d1fb6bca8e4c3210fb94fa56"} Jul 07 00:08:32.566621 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:32.566575 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" podStartSLOduration=1.5399455579999999 podStartE2EDuration="2.566558022s" totalImagesPullingTime="1.026612464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:08:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:08:32.565284749 +0000 UTC m=+678.537769411" watchObservedRunningTime="2026-07-07 00:08:32.566558022 +0000 UTC m=+678.539042684" Jul 07 00:08:40.699141 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:40.699103 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:40.699566 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:40.699163 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:40.711674 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:40.711645 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:08:41.603713 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:08:41.603680 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7" Jul 07 00:09:09.489655 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.489572 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks"] Jul 07 00:09:09.493032 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.493006 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.495632 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.495612 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 00:09:09.500733 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.500708 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks"] Jul 07 00:09:09.648788 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.648744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeb6dd09-076a-4f33-9758-dfe4b7d35237-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.648788 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.648793 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.649050 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.648843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.649050 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.648917 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mnn2\" (UniqueName: \"kubernetes.io/projected/eeb6dd09-076a-4f33-9758-dfe4b7d35237-kube-api-access-5mnn2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.649050 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.648954 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.649050 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.649019 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.750435 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.750348 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mnn2\" (UniqueName: \"kubernetes.io/projected/eeb6dd09-076a-4f33-9758-dfe4b7d35237-kube-api-access-5mnn2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.750435 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.750389 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.750670 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.750519 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.750670 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.750566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeb6dd09-076a-4f33-9758-dfe4b7d35237-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.750774 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.750705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.750774 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.750763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.750856 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.750822 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.751034 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.751009 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.751143 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.751104 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.752852 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.752829 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eeb6dd09-076a-4f33-9758-dfe4b7d35237-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.753156 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.753138 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eeb6dd09-076a-4f33-9758-dfe4b7d35237-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.758351 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.758328 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mnn2\" (UniqueName: \"kubernetes.io/projected/eeb6dd09-076a-4f33-9758-dfe4b7d35237-kube-api-access-5mnn2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6v6ks\" (UID: \"eeb6dd09-076a-4f33-9758-dfe4b7d35237\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.806029 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.805983 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:09.940554 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:09.940524 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks"] Jul 07 00:09:09.942221 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:09:09.942196 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb6dd09_076a_4f33_9758_dfe4b7d35237.slice/crio-8dede15014ced2bbe67360903321430105e52a245427a6dad3de60ab0c574598 WatchSource:0}: Error finding container 8dede15014ced2bbe67360903321430105e52a245427a6dad3de60ab0c574598: Status 404 returned error can't find the container with id 8dede15014ced2bbe67360903321430105e52a245427a6dad3de60ab0c574598 Jul 07 00:09:10.732618 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:10.732580 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" event={"ID":"eeb6dd09-076a-4f33-9758-dfe4b7d35237","Type":"ContainerStarted","Data":"d735feef2835bae7d6526495d035bffa1dc622a8d3cae537b34bcd30ab1707ea"} Jul 07 00:09:10.733000 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:10.732659 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" event={"ID":"eeb6dd09-076a-4f33-9758-dfe4b7d35237","Type":"ContainerStarted","Data":"8dede15014ced2bbe67360903321430105e52a245427a6dad3de60ab0c574598"} Jul 07 00:09:10.759035 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:10.758975 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" podStartSLOduration=1.563494148 podStartE2EDuration="1.758956492s" totalImagesPullingTime="195.462344ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:09:10.758956315 +0000 UTC m=+716.731440976" watchObservedRunningTime="2026-07-07 00:09:10.758956492 +0000 UTC m=+716.731441157" Jul 07 00:09:19.806404 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:19.806361 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:19.806877 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:19.806435 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:19.819020 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:19.818997 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:09:20.793772 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:09:20.793740 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6v6ks" Jul 07 00:11:40.068445 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.068406 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5868db4bd6-8lxdw"] Jul 07 00:11:40.068981 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.068633 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" podUID="f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8" containerName="manager" containerID="cri-o://c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff" gracePeriod=30 Jul 07 00:11:40.326505 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.326439 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:11:40.486440 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.486409 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhddg\" (UniqueName: \"kubernetes.io/projected/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-kube-api-access-jhddg\") pod \"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8\" (UID: \"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8\") " Jul 07 00:11:40.486721 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.486476 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-webhook-cert\") pod \"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8\" (UID: \"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8\") " Jul 07 00:11:40.486814 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.486752 2562 generic.go:356] "Generic (PLEG): container finished" podID="f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8" containerID="c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff" exitCode=0 Jul 07 00:11:40.486814 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.486803 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" Jul 07 00:11:40.486897 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.486839 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" event={"ID":"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8","Type":"ContainerDied","Data":"c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff"} Jul 07 00:11:40.486942 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.486907 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5868db4bd6-8lxdw" event={"ID":"f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8","Type":"ContainerDied","Data":"267464da1ca8ffe2c2b9d516f72ee9b73797b7e1f4bf0976e96b684a3b5bceed"} Jul 07 00:11:40.486942 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.486936 2562 scope.go:117] "RemoveContainer" containerID="c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff" Jul 07 00:11:40.488745 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.488712 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-kube-api-access-jhddg" (OuterVolumeSpecName: "kube-api-access-jhddg") pod "f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8" (UID: "f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8"). InnerVolumeSpecName "kube-api-access-jhddg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:11:40.488852 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.488763 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8" (UID: "f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:11:40.505670 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.505647 2562 scope.go:117] "RemoveContainer" containerID="c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff" Jul 07 00:11:40.505987 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:11:40.505968 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff\": container with ID starting with c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff not found: ID does not exist" containerID="c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff" Jul 07 00:11:40.506046 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.505997 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff"} err="failed to get container status \"c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff\": rpc error: code = NotFound desc = could not find container \"c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff\": container with ID starting with c391685648550880e87c3ab10d9b0b7f387fea02bcb78fa6baca44d05e340dff not found: ID does not exist" Jul 07 00:11:40.587449 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.587367 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhddg\" (UniqueName: \"kubernetes.io/projected/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-kube-api-access-jhddg\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:11:40.587449 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.587395 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8-webhook-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:11:40.804209 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.804170 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5868db4bd6-8lxdw"] Jul 07 00:11:40.811493 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:40.811461 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5868db4bd6-8lxdw"] Jul 07 00:11:41.620341 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.620304 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5868db4bd6-qvcsh"] Jul 07 00:11:41.620839 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.620761 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8" containerName="manager" Jul 07 00:11:41.620839 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.620780 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8" containerName="manager" Jul 07 00:11:41.620945 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.620890 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8" containerName="manager" Jul 07 00:11:41.625031 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.625012 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5868db4bd6-qvcsh" Jul 07 00:11:41.628376 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.628357 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 00:11:41.629315 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.629288 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-thk6q\"" Jul 07 00:11:41.629435 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.629319 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 00:11:41.635180 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.635155 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5868db4bd6-qvcsh"] Jul 07 00:11:41.696150 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.696090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6gwg\" (UniqueName: \"kubernetes.io/projected/8623a11e-31f3-40aa-b164-3781d41596e0-kube-api-access-q6gwg\") pod \"maas-controller-5868db4bd6-qvcsh\" (UID: \"8623a11e-31f3-40aa-b164-3781d41596e0\") " pod="opendatahub/maas-controller-5868db4bd6-qvcsh" Jul 07 00:11:41.696150 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.696165 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8623a11e-31f3-40aa-b164-3781d41596e0-webhook-cert\") pod \"maas-controller-5868db4bd6-qvcsh\" (UID: \"8623a11e-31f3-40aa-b164-3781d41596e0\") " pod="opendatahub/maas-controller-5868db4bd6-qvcsh" Jul 07 00:11:41.797430 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.797392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6gwg\" (UniqueName: \"kubernetes.io/projected/8623a11e-31f3-40aa-b164-3781d41596e0-kube-api-access-q6gwg\") pod \"maas-controller-5868db4bd6-qvcsh\" (UID: \"8623a11e-31f3-40aa-b164-3781d41596e0\") " pod="opendatahub/maas-controller-5868db4bd6-qvcsh" Jul 07 00:11:41.797550 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.797440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8623a11e-31f3-40aa-b164-3781d41596e0-webhook-cert\") pod \"maas-controller-5868db4bd6-qvcsh\" (UID: \"8623a11e-31f3-40aa-b164-3781d41596e0\") " pod="opendatahub/maas-controller-5868db4bd6-qvcsh" Jul 07 00:11:41.800143 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.800106 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8623a11e-31f3-40aa-b164-3781d41596e0-webhook-cert\") pod \"maas-controller-5868db4bd6-qvcsh\" (UID: \"8623a11e-31f3-40aa-b164-3781d41596e0\") " pod="opendatahub/maas-controller-5868db4bd6-qvcsh" Jul 07 00:11:41.806056 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.806030 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6gwg\" (UniqueName: \"kubernetes.io/projected/8623a11e-31f3-40aa-b164-3781d41596e0-kube-api-access-q6gwg\") pod \"maas-controller-5868db4bd6-qvcsh\" (UID: \"8623a11e-31f3-40aa-b164-3781d41596e0\") " pod="opendatahub/maas-controller-5868db4bd6-qvcsh" Jul 07 00:11:41.937206 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:41.937116 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5868db4bd6-qvcsh" Jul 07 00:11:42.073946 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:42.073920 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5868db4bd6-qvcsh"] Jul 07 00:11:42.075774 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:11:42.075740 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8623a11e_31f3_40aa_b164_3781d41596e0.slice/crio-8b54917927e43152e21f7de318a9af61c1f2b29af0590814e780edf0eea75afe WatchSource:0}: Error finding container 8b54917927e43152e21f7de318a9af61c1f2b29af0590814e780edf0eea75afe: Status 404 returned error can't find the container with id 8b54917927e43152e21f7de318a9af61c1f2b29af0590814e780edf0eea75afe Jul 07 00:11:42.502792 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:42.502753 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5868db4bd6-qvcsh" event={"ID":"8623a11e-31f3-40aa-b164-3781d41596e0","Type":"ContainerStarted","Data":"8b54917927e43152e21f7de318a9af61c1f2b29af0590814e780edf0eea75afe"} Jul 07 00:11:42.582933 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:42.582896 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8" path="/var/lib/kubelet/pods/f6dc75d2-5a01-4af1-8b23-69cd3aa4a0b8/volumes" Jul 07 00:11:43.509053 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:43.509017 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5868db4bd6-qvcsh" event={"ID":"8623a11e-31f3-40aa-b164-3781d41596e0","Type":"ContainerStarted","Data":"2bc0f6884428b24b4accf754ba998361140cf555c8fb9a223f32d5f115dd386b"} Jul 07 00:11:43.509458 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:43.509221 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5868db4bd6-qvcsh" Jul 07 00:11:43.543304 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:43.543246 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5868db4bd6-qvcsh" podStartSLOduration=1.977537833 podStartE2EDuration="2.54322899s" totalImagesPullingTime="565.691157ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:11:43.54195577 +0000 UTC m=+869.514440430" watchObservedRunningTime="2026-07-07 00:11:43.54322899 +0000 UTC m=+869.515713651" Jul 07 00:11:54.519843 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:11:54.519807 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5868db4bd6-qvcsh" Jul 07 00:23:48.842814 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:48.842779 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk"] Jul 07 00:23:48.846415 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:48.846393 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:48.854033 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:48.854009 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-58bac345-serving-cert\"" Jul 07 00:23:48.867121 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:48.867087 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk"] Jul 07 00:23:48.935401 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:48.935363 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/524814fd-0a2f-402d-b29f-389364203c9c-openshift-service-ca\") pod \"maas-api-e2e-ait-58bac345-586fc94f96-z9pjk\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:48.935401 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:48.935411 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/524814fd-0a2f-402d-b29f-389364203c9c-maas-api-tls\") pod \"maas-api-e2e-ait-58bac345-586fc94f96-z9pjk\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:48.935638 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:48.935457 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcmxc\" (UniqueName: \"kubernetes.io/projected/524814fd-0a2f-402d-b29f-389364203c9c-kube-api-access-rcmxc\") pod \"maas-api-e2e-ait-58bac345-586fc94f96-z9pjk\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:49.036252 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:49.036213 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/524814fd-0a2f-402d-b29f-389364203c9c-openshift-service-ca\") pod \"maas-api-e2e-ait-58bac345-586fc94f96-z9pjk\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:49.036453 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:49.036265 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/524814fd-0a2f-402d-b29f-389364203c9c-maas-api-tls\") pod \"maas-api-e2e-ait-58bac345-586fc94f96-z9pjk\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:49.036453 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:49.036309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcmxc\" (UniqueName: \"kubernetes.io/projected/524814fd-0a2f-402d-b29f-389364203c9c-kube-api-access-rcmxc\") pod \"maas-api-e2e-ait-58bac345-586fc94f96-z9pjk\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:49.036943 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:49.036917 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/524814fd-0a2f-402d-b29f-389364203c9c-openshift-service-ca\") pod \"maas-api-e2e-ait-58bac345-586fc94f96-z9pjk\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:49.038620 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:49.038600 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/524814fd-0a2f-402d-b29f-389364203c9c-maas-api-tls\") pod \"maas-api-e2e-ait-58bac345-586fc94f96-z9pjk\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:49.047611 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:49.047581 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcmxc\" (UniqueName: \"kubernetes.io/projected/524814fd-0a2f-402d-b29f-389364203c9c-kube-api-access-rcmxc\") pod \"maas-api-e2e-ait-58bac345-586fc94f96-z9pjk\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:49.157873 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:49.157835 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:49.304226 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:49.304194 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk"] Jul 07 00:23:49.306237 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:23:49.306212 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod524814fd_0a2f_402d_b29f_389364203c9c.slice/crio-7edd46c056b63697967cc2335d2940691208022a3a27b9f01467c7faea6a18c7 WatchSource:0}: Error finding container 7edd46c056b63697967cc2335d2940691208022a3a27b9f01467c7faea6a18c7: Status 404 returned error can't find the container with id 7edd46c056b63697967cc2335d2940691208022a3a27b9f01467c7faea6a18c7 Jul 07 00:23:49.307764 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:49.307745 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 00:23:50.153339 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:50.153302 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" event={"ID":"524814fd-0a2f-402d-b29f-389364203c9c","Type":"ContainerStarted","Data":"4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee"} Jul 07 00:23:50.153750 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:50.153366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" event={"ID":"524814fd-0a2f-402d-b29f-389364203c9c","Type":"ContainerStarted","Data":"7edd46c056b63697967cc2335d2940691208022a3a27b9f01467c7faea6a18c7"} Jul 07 00:23:50.153750 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:50.153408 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:50.172544 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:50.172439 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" podStartSLOduration=1.5574839379999998 podStartE2EDuration="2.172419712s" totalImagesPullingTime="614.935774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:23:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:23:50.1701996 +0000 UTC m=+1596.142684263" watchObservedRunningTime="2026-07-07 00:23:50.172419712 +0000 UTC m=+1596.144904373" Jul 07 00:23:54.346947 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.346902 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk"] Jul 07 00:23:54.347461 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.347155 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" podUID="524814fd-0a2f-402d-b29f-389364203c9c" containerName="maas-api" containerID="cri-o://4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee" gracePeriod=30 Jul 07 00:23:54.353580 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.353556 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:54.603327 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.603265 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:54.687693 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.687653 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/524814fd-0a2f-402d-b29f-389364203c9c-maas-api-tls\") pod \"524814fd-0a2f-402d-b29f-389364203c9c\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " Jul 07 00:23:54.687899 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.687715 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcmxc\" (UniqueName: \"kubernetes.io/projected/524814fd-0a2f-402d-b29f-389364203c9c-kube-api-access-rcmxc\") pod \"524814fd-0a2f-402d-b29f-389364203c9c\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " Jul 07 00:23:54.687899 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.687769 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/524814fd-0a2f-402d-b29f-389364203c9c-openshift-service-ca\") pod \"524814fd-0a2f-402d-b29f-389364203c9c\" (UID: \"524814fd-0a2f-402d-b29f-389364203c9c\") " Jul 07 00:23:54.688247 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.688219 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/524814fd-0a2f-402d-b29f-389364203c9c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "524814fd-0a2f-402d-b29f-389364203c9c" (UID: "524814fd-0a2f-402d-b29f-389364203c9c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:23:54.690035 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.690006 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/524814fd-0a2f-402d-b29f-389364203c9c-kube-api-access-rcmxc" (OuterVolumeSpecName: "kube-api-access-rcmxc") pod "524814fd-0a2f-402d-b29f-389364203c9c" (UID: "524814fd-0a2f-402d-b29f-389364203c9c"). InnerVolumeSpecName "kube-api-access-rcmxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:23:54.690035 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.690009 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/524814fd-0a2f-402d-b29f-389364203c9c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "524814fd-0a2f-402d-b29f-389364203c9c" (UID: "524814fd-0a2f-402d-b29f-389364203c9c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:23:54.789396 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.789356 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcmxc\" (UniqueName: \"kubernetes.io/projected/524814fd-0a2f-402d-b29f-389364203c9c-kube-api-access-rcmxc\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:23:54.789396 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.789391 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/524814fd-0a2f-402d-b29f-389364203c9c-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:23:54.789396 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:54.789402 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/524814fd-0a2f-402d-b29f-389364203c9c-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:23:55.179863 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:55.179829 2562 generic.go:356] "Generic (PLEG): container finished" podID="524814fd-0a2f-402d-b29f-389364203c9c" containerID="4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee" exitCode=0 Jul 07 00:23:55.180080 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:55.179883 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" event={"ID":"524814fd-0a2f-402d-b29f-389364203c9c","Type":"ContainerDied","Data":"4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee"} Jul 07 00:23:55.180080 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:55.179886 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" Jul 07 00:23:55.180080 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:55.179925 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk" event={"ID":"524814fd-0a2f-402d-b29f-389364203c9c","Type":"ContainerDied","Data":"7edd46c056b63697967cc2335d2940691208022a3a27b9f01467c7faea6a18c7"} Jul 07 00:23:55.180080 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:55.179946 2562 scope.go:117] "RemoveContainer" containerID="4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee" Jul 07 00:23:55.189395 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:55.189379 2562 scope.go:117] "RemoveContainer" containerID="4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee" Jul 07 00:23:55.189680 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:23:55.189658 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee\": container with ID starting with 4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee not found: ID does not exist" containerID="4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee" Jul 07 00:23:55.189741 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:55.189688 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee"} err="failed to get container status \"4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee\": rpc error: code = NotFound desc = could not find container \"4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee\": container with ID starting with 4427d9ae443842a64e30353a5ce1a95f648b562fb4ad4528e6c48c47b03225ee not found: ID does not exist" Jul 07 00:23:55.206419 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:55.206385 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk"] Jul 07 00:23:55.211560 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:55.211533 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58bac345-586fc94f96-z9pjk"] Jul 07 00:23:56.583181 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:23:56.583143 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="524814fd-0a2f-402d-b29f-389364203c9c" path="/var/lib/kubelet/pods/524814fd-0a2f-402d-b29f-389364203c9c/volumes" Jul 07 00:24:01.474811 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.474762 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj"] Jul 07 00:24:01.475327 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.475260 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="524814fd-0a2f-402d-b29f-389364203c9c" containerName="maas-api" Jul 07 00:24:01.475327 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.475278 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="524814fd-0a2f-402d-b29f-389364203c9c" containerName="maas-api" Jul 07 00:24:01.475437 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.475360 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="524814fd-0a2f-402d-b29f-389364203c9c" containerName="maas-api" Jul 07 00:24:01.480495 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.480467 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:01.482956 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.482934 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6738905c-serving-cert\"" Jul 07 00:24:01.489952 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.489922 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj"] Jul 07 00:24:01.551305 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.551262 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902ccaf-0471-48a2-8c53-dc0564d7034d-maas-api-tls\") pod \"maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:01.551305 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.551306 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c902ccaf-0471-48a2-8c53-dc0564d7034d-openshift-service-ca\") pod \"maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:01.551534 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.551334 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7bxh\" (UniqueName: \"kubernetes.io/projected/c902ccaf-0471-48a2-8c53-dc0564d7034d-kube-api-access-b7bxh\") pod \"maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:01.652686 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.652652 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902ccaf-0471-48a2-8c53-dc0564d7034d-maas-api-tls\") pod \"maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:01.652686 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.652688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c902ccaf-0471-48a2-8c53-dc0564d7034d-openshift-service-ca\") pod \"maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:01.652950 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.652723 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7bxh\" (UniqueName: \"kubernetes.io/projected/c902ccaf-0471-48a2-8c53-dc0564d7034d-kube-api-access-b7bxh\") pod \"maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:01.652950 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:24:01.652814 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-serving-cert: secret "maas-api-e2e-ait-6738905c-serving-cert" not found Jul 07 00:24:01.652950 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:24:01.652894 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c902ccaf-0471-48a2-8c53-dc0564d7034d-maas-api-tls podName:c902ccaf-0471-48a2-8c53-dc0564d7034d nodeName:}" failed. No retries permitted until 2026-07-07 00:24:02.152871186 +0000 UTC m=+1608.125355826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c902ccaf-0471-48a2-8c53-dc0564d7034d-maas-api-tls") pod "maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" (UID: "c902ccaf-0471-48a2-8c53-dc0564d7034d") : secret "maas-api-e2e-ait-6738905c-serving-cert" not found Jul 07 00:24:01.653391 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.653372 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c902ccaf-0471-48a2-8c53-dc0564d7034d-openshift-service-ca\") pod \"maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:01.663045 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:01.663018 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7bxh\" (UniqueName: \"kubernetes.io/projected/c902ccaf-0471-48a2-8c53-dc0564d7034d-kube-api-access-b7bxh\") pod \"maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:02.157054 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:02.157015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902ccaf-0471-48a2-8c53-dc0564d7034d-maas-api-tls\") pod \"maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:02.159632 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:02.159604 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902ccaf-0471-48a2-8c53-dc0564d7034d-maas-api-tls\") pod \"maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:02.395397 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:02.395365 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:02.533181 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:02.533154 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj"] Jul 07 00:24:02.535454 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:24:02.535429 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc902ccaf_0471_48a2_8c53_dc0564d7034d.slice/crio-580aa4c2cab8ebe3f0bad358b9512612b938b983c6b6cdb263b1b697aee93fe6 WatchSource:0}: Error finding container 580aa4c2cab8ebe3f0bad358b9512612b938b983c6b6cdb263b1b697aee93fe6: Status 404 returned error can't find the container with id 580aa4c2cab8ebe3f0bad358b9512612b938b983c6b6cdb263b1b697aee93fe6 Jul 07 00:24:03.222852 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:03.222813 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" event={"ID":"c902ccaf-0471-48a2-8c53-dc0564d7034d","Type":"ContainerStarted","Data":"580aa4c2cab8ebe3f0bad358b9512612b938b983c6b6cdb263b1b697aee93fe6"} Jul 07 00:24:04.230427 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:04.230386 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" event={"ID":"c902ccaf-0471-48a2-8c53-dc0564d7034d","Type":"ContainerStarted","Data":"b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb"} Jul 07 00:24:04.230840 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:04.230482 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:04.248265 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:04.248210 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" podStartSLOduration=2.603934427 podStartE2EDuration="3.248194091s" totalImagesPullingTime="644.259664ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:24:04.247404058 +0000 UTC m=+1610.219888720" watchObservedRunningTime="2026-07-07 00:24:04.248194091 +0000 UTC m=+1610.220678752" Jul 07 00:24:06.944503 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:06.944416 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj"] Jul 07 00:24:06.944945 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:06.944657 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" podUID="c902ccaf-0471-48a2-8c53-dc0564d7034d" containerName="maas-api" containerID="cri-o://b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb" gracePeriod=30 Jul 07 00:24:07.194246 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.194219 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:07.246224 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.246132 2562 generic.go:356] "Generic (PLEG): container finished" podID="c902ccaf-0471-48a2-8c53-dc0564d7034d" containerID="b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb" exitCode=0 Jul 07 00:24:07.246224 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.246181 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" Jul 07 00:24:07.246453 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.246217 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" event={"ID":"c902ccaf-0471-48a2-8c53-dc0564d7034d","Type":"ContainerDied","Data":"b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb"} Jul 07 00:24:07.246453 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.246281 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj" event={"ID":"c902ccaf-0471-48a2-8c53-dc0564d7034d","Type":"ContainerDied","Data":"580aa4c2cab8ebe3f0bad358b9512612b938b983c6b6cdb263b1b697aee93fe6"} Jul 07 00:24:07.246453 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.246311 2562 scope.go:117] "RemoveContainer" containerID="b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb" Jul 07 00:24:07.254727 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.254708 2562 scope.go:117] "RemoveContainer" containerID="b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb" Jul 07 00:24:07.254997 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:24:07.254978 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb\": container with ID starting with b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb not found: ID does not exist" containerID="b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb" Jul 07 00:24:07.255039 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.255009 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb"} err="failed to get container status \"b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb\": rpc error: code = NotFound desc = could not find container \"b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb\": container with ID starting with b43d2c5c6153cd6e74663cdc13f29961cc42404dd27ed6345bc03fcac8871bcb not found: ID does not exist" Jul 07 00:24:07.303779 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.303746 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7bxh\" (UniqueName: \"kubernetes.io/projected/c902ccaf-0471-48a2-8c53-dc0564d7034d-kube-api-access-b7bxh\") pod \"c902ccaf-0471-48a2-8c53-dc0564d7034d\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " Jul 07 00:24:07.303945 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.303797 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902ccaf-0471-48a2-8c53-dc0564d7034d-maas-api-tls\") pod \"c902ccaf-0471-48a2-8c53-dc0564d7034d\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " Jul 07 00:24:07.303945 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.303843 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c902ccaf-0471-48a2-8c53-dc0564d7034d-openshift-service-ca\") pod \"c902ccaf-0471-48a2-8c53-dc0564d7034d\" (UID: \"c902ccaf-0471-48a2-8c53-dc0564d7034d\") " Jul 07 00:24:07.304254 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.304230 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c902ccaf-0471-48a2-8c53-dc0564d7034d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c902ccaf-0471-48a2-8c53-dc0564d7034d" (UID: "c902ccaf-0471-48a2-8c53-dc0564d7034d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:24:07.305896 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.305868 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c902ccaf-0471-48a2-8c53-dc0564d7034d-kube-api-access-b7bxh" (OuterVolumeSpecName: "kube-api-access-b7bxh") pod "c902ccaf-0471-48a2-8c53-dc0564d7034d" (UID: "c902ccaf-0471-48a2-8c53-dc0564d7034d"). InnerVolumeSpecName "kube-api-access-b7bxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:24:07.305896 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.305880 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c902ccaf-0471-48a2-8c53-dc0564d7034d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c902ccaf-0471-48a2-8c53-dc0564d7034d" (UID: "c902ccaf-0471-48a2-8c53-dc0564d7034d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:24:07.405299 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.405262 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c902ccaf-0471-48a2-8c53-dc0564d7034d-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:24:07.405299 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.405295 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c902ccaf-0471-48a2-8c53-dc0564d7034d-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:24:07.405299 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.405307 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7bxh\" (UniqueName: \"kubernetes.io/projected/c902ccaf-0471-48a2-8c53-dc0564d7034d-kube-api-access-b7bxh\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:24:07.568851 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.568821 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj"] Jul 07 00:24:07.577962 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:07.577933 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6738905c-558cb4cd64-rnjfj"] Jul 07 00:24:08.582927 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:08.582891 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c902ccaf-0471-48a2-8c53-dc0564d7034d" path="/var/lib/kubelet/pods/c902ccaf-0471-48a2-8c53-dc0564d7034d/volumes" Jul 07 00:24:19.955816 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:19.955780 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr"] Jul 07 00:24:19.956253 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:19.956147 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c902ccaf-0471-48a2-8c53-dc0564d7034d" containerName="maas-api" Jul 07 00:24:19.956253 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:19.956160 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c902ccaf-0471-48a2-8c53-dc0564d7034d" containerName="maas-api" Jul 07 00:24:19.956253 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:19.956228 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c902ccaf-0471-48a2-8c53-dc0564d7034d" containerName="maas-api" Jul 07 00:24:19.960801 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:19.960768 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:19.963271 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:19.963247 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-6c06fb72-serving-cert\"" Jul 07 00:24:19.969316 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:19.969214 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr"] Jul 07 00:24:20.120836 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:20.120803 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r77vf\" (UniqueName: \"kubernetes.io/projected/60da85a1-3c7b-4fd4-a33b-72d890dfd410-kube-api-access-r77vf\") pod \"maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:20.120836 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:20.120842 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60da85a1-3c7b-4fd4-a33b-72d890dfd410-maas-api-tls\") pod \"maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:20.121053 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:20.120887 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60da85a1-3c7b-4fd4-a33b-72d890dfd410-openshift-service-ca\") pod \"maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:20.222280 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:20.222178 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60da85a1-3c7b-4fd4-a33b-72d890dfd410-openshift-service-ca\") pod \"maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:20.222461 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:20.222299 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r77vf\" (UniqueName: \"kubernetes.io/projected/60da85a1-3c7b-4fd4-a33b-72d890dfd410-kube-api-access-r77vf\") pod \"maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:20.222461 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:20.222329 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60da85a1-3c7b-4fd4-a33b-72d890dfd410-maas-api-tls\") pod \"maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:20.222893 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:20.222868 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60da85a1-3c7b-4fd4-a33b-72d890dfd410-openshift-service-ca\") pod \"maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:20.224704 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:20.224685 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60da85a1-3c7b-4fd4-a33b-72d890dfd410-maas-api-tls\") pod \"maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:20.231181 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:20.231161 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r77vf\" (UniqueName: \"kubernetes.io/projected/60da85a1-3c7b-4fd4-a33b-72d890dfd410-kube-api-access-r77vf\") pod \"maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:20.275163 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:20.275128 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:20.414220 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:20.414191 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr"] Jul 07 00:24:20.416407 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:24:20.416373 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60da85a1_3c7b_4fd4_a33b_72d890dfd410.slice/crio-d0eaefc09b45a4190fe0854084fbfe55967c5a6db696fffc5d4b36066bb8a80a WatchSource:0}: Error finding container d0eaefc09b45a4190fe0854084fbfe55967c5a6db696fffc5d4b36066bb8a80a: Status 404 returned error can't find the container with id d0eaefc09b45a4190fe0854084fbfe55967c5a6db696fffc5d4b36066bb8a80a Jul 07 00:24:21.320951 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:21.320904 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" event={"ID":"60da85a1-3c7b-4fd4-a33b-72d890dfd410","Type":"ContainerStarted","Data":"62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa"} Jul 07 00:24:21.321443 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:21.320976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" event={"ID":"60da85a1-3c7b-4fd4-a33b-72d890dfd410","Type":"ContainerStarted","Data":"d0eaefc09b45a4190fe0854084fbfe55967c5a6db696fffc5d4b36066bb8a80a"} Jul 07 00:24:21.321443 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:21.321141 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:21.340287 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:21.340237 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" podStartSLOduration=2.015429552 podStartE2EDuration="2.340219701s" totalImagesPullingTime="324.790149ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:24:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:24:21.337947577 +0000 UTC m=+1627.310432245" watchObservedRunningTime="2026-07-07 00:24:21.340219701 +0000 UTC m=+1627.312704361" Jul 07 00:24:24.735639 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:24.735598 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr"] Jul 07 00:24:24.736170 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:24.735943 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" podUID="60da85a1-3c7b-4fd4-a33b-72d890dfd410" containerName="maas-api" containerID="cri-o://62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa" gracePeriod=30 Jul 07 00:24:24.987251 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:24.987173 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:25.168855 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.168804 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60da85a1-3c7b-4fd4-a33b-72d890dfd410-maas-api-tls\") pod \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " Jul 07 00:24:25.169101 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.168931 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60da85a1-3c7b-4fd4-a33b-72d890dfd410-openshift-service-ca\") pod \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " Jul 07 00:24:25.169101 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.169004 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r77vf\" (UniqueName: \"kubernetes.io/projected/60da85a1-3c7b-4fd4-a33b-72d890dfd410-kube-api-access-r77vf\") pod \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\" (UID: \"60da85a1-3c7b-4fd4-a33b-72d890dfd410\") " Jul 07 00:24:25.169302 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.169277 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60da85a1-3c7b-4fd4-a33b-72d890dfd410-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "60da85a1-3c7b-4fd4-a33b-72d890dfd410" (UID: "60da85a1-3c7b-4fd4-a33b-72d890dfd410"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:24:25.171051 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.171028 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60da85a1-3c7b-4fd4-a33b-72d890dfd410-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "60da85a1-3c7b-4fd4-a33b-72d890dfd410" (UID: "60da85a1-3c7b-4fd4-a33b-72d890dfd410"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:24:25.171319 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.171284 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60da85a1-3c7b-4fd4-a33b-72d890dfd410-kube-api-access-r77vf" (OuterVolumeSpecName: "kube-api-access-r77vf") pod "60da85a1-3c7b-4fd4-a33b-72d890dfd410" (UID: "60da85a1-3c7b-4fd4-a33b-72d890dfd410"). InnerVolumeSpecName "kube-api-access-r77vf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:24:25.269906 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.269820 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r77vf\" (UniqueName: \"kubernetes.io/projected/60da85a1-3c7b-4fd4-a33b-72d890dfd410-kube-api-access-r77vf\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:24:25.269906 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.269849 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60da85a1-3c7b-4fd4-a33b-72d890dfd410-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:24:25.269906 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.269858 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60da85a1-3c7b-4fd4-a33b-72d890dfd410-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:24:25.344379 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.344341 2562 generic.go:356] "Generic (PLEG): container finished" podID="60da85a1-3c7b-4fd4-a33b-72d890dfd410" containerID="62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa" exitCode=0 Jul 07 00:24:25.344556 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.344389 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" Jul 07 00:24:25.344556 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.344416 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" event={"ID":"60da85a1-3c7b-4fd4-a33b-72d890dfd410","Type":"ContainerDied","Data":"62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa"} Jul 07 00:24:25.344556 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.344463 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr" event={"ID":"60da85a1-3c7b-4fd4-a33b-72d890dfd410","Type":"ContainerDied","Data":"d0eaefc09b45a4190fe0854084fbfe55967c5a6db696fffc5d4b36066bb8a80a"} Jul 07 00:24:25.344556 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.344485 2562 scope.go:117] "RemoveContainer" containerID="62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa" Jul 07 00:24:25.353607 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.353589 2562 scope.go:117] "RemoveContainer" containerID="62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa" Jul 07 00:24:25.353886 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:24:25.353864 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa\": container with ID starting with 62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa not found: ID does not exist" containerID="62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa" Jul 07 00:24:25.353931 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.353895 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa"} err="failed to get container status \"62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa\": rpc error: code = NotFound desc = could not find container \"62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa\": container with ID starting with 62f9ae8b78e75aa3a9d61cb2c9a79ab57ba47d72c5be0055edc443ebb635edaa not found: ID does not exist" Jul 07 00:24:25.366641 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.366608 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr"] Jul 07 00:24:25.368761 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:25.368723 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c06fb72-5969bbfb74-j7sjr"] Jul 07 00:24:26.582750 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:24:26.582667 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60da85a1-3c7b-4fd4-a33b-72d890dfd410" path="/var/lib/kubelet/pods/60da85a1-3c7b-4fd4-a33b-72d890dfd410/volumes" Jul 07 00:26:10.635645 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.635549 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf"] Jul 07 00:26:10.636251 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.636128 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60da85a1-3c7b-4fd4-a33b-72d890dfd410" containerName="maas-api" Jul 07 00:26:10.636251 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.636149 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="60da85a1-3c7b-4fd4-a33b-72d890dfd410" containerName="maas-api" Jul 07 00:26:10.636379 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.636259 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="60da85a1-3c7b-4fd4-a33b-72d890dfd410" containerName="maas-api" Jul 07 00:26:10.639309 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.639287 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:10.642725 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.642705 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9eadf20a-serving-cert\"" Jul 07 00:26:10.666077 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.666042 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf"] Jul 07 00:26:10.813449 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.813414 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caac2bd3-a43c-4d19-88d9-2a70d6aca770-openshift-service-ca\") pod \"maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:10.813684 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.813469 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caac2bd3-a43c-4d19-88d9-2a70d6aca770-maas-api-tls\") pod \"maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:10.813684 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.813576 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm674\" (UniqueName: \"kubernetes.io/projected/caac2bd3-a43c-4d19-88d9-2a70d6aca770-kube-api-access-vm674\") pod \"maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:10.914532 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.914437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caac2bd3-a43c-4d19-88d9-2a70d6aca770-openshift-service-ca\") pod \"maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:10.914532 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.914495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caac2bd3-a43c-4d19-88d9-2a70d6aca770-maas-api-tls\") pod \"maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:10.914739 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.914570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm674\" (UniqueName: \"kubernetes.io/projected/caac2bd3-a43c-4d19-88d9-2a70d6aca770-kube-api-access-vm674\") pod \"maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:10.915240 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.915216 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caac2bd3-a43c-4d19-88d9-2a70d6aca770-openshift-service-ca\") pod \"maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:10.917348 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.917322 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caac2bd3-a43c-4d19-88d9-2a70d6aca770-maas-api-tls\") pod \"maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:10.924412 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.924385 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm674\" (UniqueName: \"kubernetes.io/projected/caac2bd3-a43c-4d19-88d9-2a70d6aca770-kube-api-access-vm674\") pod \"maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:10.949395 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:10.949358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:11.095684 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:11.095658 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf"] Jul 07 00:26:11.097120 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:26:11.097090 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaac2bd3_a43c_4d19_88d9_2a70d6aca770.slice/crio-4b01683733b9a251a36d3dc696f7cc53ef1110290573be3634ab37989c0eeca9 WatchSource:0}: Error finding container 4b01683733b9a251a36d3dc696f7cc53ef1110290573be3634ab37989c0eeca9: Status 404 returned error can't find the container with id 4b01683733b9a251a36d3dc696f7cc53ef1110290573be3634ab37989c0eeca9 Jul 07 00:26:11.889520 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:11.889482 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" event={"ID":"caac2bd3-a43c-4d19-88d9-2a70d6aca770","Type":"ContainerStarted","Data":"1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f"} Jul 07 00:26:11.889977 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:11.889556 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:11.889977 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:11.889582 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" event={"ID":"caac2bd3-a43c-4d19-88d9-2a70d6aca770","Type":"ContainerStarted","Data":"4b01683733b9a251a36d3dc696f7cc53ef1110290573be3634ab37989c0eeca9"} Jul 07 00:26:11.909548 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:11.909495 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" podStartSLOduration=1.430821369 podStartE2EDuration="1.909479237s" totalImagesPullingTime="478.657868ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:26:11.908041811 +0000 UTC m=+1737.880526485" watchObservedRunningTime="2026-07-07 00:26:11.909479237 +0000 UTC m=+1737.881963919" Jul 07 00:26:17.900631 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:17.900550 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:22.937095 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:22.937050 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf"] Jul 07 00:26:22.937543 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:22.937257 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" podUID="caac2bd3-a43c-4d19-88d9-2a70d6aca770" containerName="maas-api" containerID="cri-o://1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f" gracePeriod=30 Jul 07 00:26:23.188476 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.188406 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:23.220596 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.220567 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caac2bd3-a43c-4d19-88d9-2a70d6aca770-openshift-service-ca\") pod \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " Jul 07 00:26:23.220762 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.220613 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caac2bd3-a43c-4d19-88d9-2a70d6aca770-maas-api-tls\") pod \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " Jul 07 00:26:23.220762 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.220665 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm674\" (UniqueName: \"kubernetes.io/projected/caac2bd3-a43c-4d19-88d9-2a70d6aca770-kube-api-access-vm674\") pod \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\" (UID: \"caac2bd3-a43c-4d19-88d9-2a70d6aca770\") " Jul 07 00:26:23.221050 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.221021 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caac2bd3-a43c-4d19-88d9-2a70d6aca770-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "caac2bd3-a43c-4d19-88d9-2a70d6aca770" (UID: "caac2bd3-a43c-4d19-88d9-2a70d6aca770"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:26:23.222920 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.222891 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caac2bd3-a43c-4d19-88d9-2a70d6aca770-kube-api-access-vm674" (OuterVolumeSpecName: "kube-api-access-vm674") pod "caac2bd3-a43c-4d19-88d9-2a70d6aca770" (UID: "caac2bd3-a43c-4d19-88d9-2a70d6aca770"). InnerVolumeSpecName "kube-api-access-vm674". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:26:23.223039 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.222948 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caac2bd3-a43c-4d19-88d9-2a70d6aca770-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "caac2bd3-a43c-4d19-88d9-2a70d6aca770" (UID: "caac2bd3-a43c-4d19-88d9-2a70d6aca770"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:26:23.322445 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.322406 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm674\" (UniqueName: \"kubernetes.io/projected/caac2bd3-a43c-4d19-88d9-2a70d6aca770-kube-api-access-vm674\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:26:23.322445 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.322436 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caac2bd3-a43c-4d19-88d9-2a70d6aca770-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:26:23.322445 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.322449 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caac2bd3-a43c-4d19-88d9-2a70d6aca770-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:26:23.953704 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.953669 2562 generic.go:356] "Generic (PLEG): container finished" podID="caac2bd3-a43c-4d19-88d9-2a70d6aca770" containerID="1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f" exitCode=0 Jul 07 00:26:23.954143 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.953725 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" Jul 07 00:26:23.954143 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.953747 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" event={"ID":"caac2bd3-a43c-4d19-88d9-2a70d6aca770","Type":"ContainerDied","Data":"1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f"} Jul 07 00:26:23.954143 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.953810 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf" event={"ID":"caac2bd3-a43c-4d19-88d9-2a70d6aca770","Type":"ContainerDied","Data":"4b01683733b9a251a36d3dc696f7cc53ef1110290573be3634ab37989c0eeca9"} Jul 07 00:26:23.954143 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.953832 2562 scope.go:117] "RemoveContainer" containerID="1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f" Jul 07 00:26:23.963193 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.963176 2562 scope.go:117] "RemoveContainer" containerID="1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f" Jul 07 00:26:23.963487 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:26:23.963462 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f\": container with ID starting with 1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f not found: ID does not exist" containerID="1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f" Jul 07 00:26:23.963541 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.963496 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f"} err="failed to get container status \"1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f\": rpc error: code = NotFound desc = could not find container \"1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f\": container with ID starting with 1884142fc295465483d52e75b8b2953630ad3de046e63db377ae8e842d77414f not found: ID does not exist" Jul 07 00:26:23.978902 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.978870 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf"] Jul 07 00:26:23.985642 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:23.985613 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9eadf20a-86dc68db56-g82nf"] Jul 07 00:26:24.584551 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:24.584521 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caac2bd3-a43c-4d19-88d9-2a70d6aca770" path="/var/lib/kubelet/pods/caac2bd3-a43c-4d19-88d9-2a70d6aca770/volumes" Jul 07 00:26:51.570574 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.570538 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482"] Jul 07 00:26:51.571183 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.570993 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caac2bd3-a43c-4d19-88d9-2a70d6aca770" containerName="maas-api" Jul 07 00:26:51.571183 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.571007 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="caac2bd3-a43c-4d19-88d9-2a70d6aca770" containerName="maas-api" Jul 07 00:26:51.571183 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.571059 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="caac2bd3-a43c-4d19-88d9-2a70d6aca770" containerName="maas-api" Jul 07 00:26:51.581315 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.581286 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:51.585146 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.585107 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-efed8c96-serving-cert\"" Jul 07 00:26:51.585861 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.585826 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482"] Jul 07 00:26:51.675560 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.675522 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szgg9\" (UniqueName: \"kubernetes.io/projected/85af738b-2226-4de6-a1ff-443616b582d9-kube-api-access-szgg9\") pod \"maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:51.675712 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.675614 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85af738b-2226-4de6-a1ff-443616b582d9-maas-api-tls\") pod \"maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:51.675785 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.675718 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85af738b-2226-4de6-a1ff-443616b582d9-openshift-service-ca\") pod \"maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:51.776759 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.776720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85af738b-2226-4de6-a1ff-443616b582d9-openshift-service-ca\") pod \"maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:51.776929 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.776771 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szgg9\" (UniqueName: \"kubernetes.io/projected/85af738b-2226-4de6-a1ff-443616b582d9-kube-api-access-szgg9\") pod \"maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:51.776929 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.776803 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85af738b-2226-4de6-a1ff-443616b582d9-maas-api-tls\") pod \"maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:51.777500 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.777475 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85af738b-2226-4de6-a1ff-443616b582d9-openshift-service-ca\") pod \"maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:51.779417 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.779396 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85af738b-2226-4de6-a1ff-443616b582d9-maas-api-tls\") pod \"maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:51.787325 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.787294 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szgg9\" (UniqueName: \"kubernetes.io/projected/85af738b-2226-4de6-a1ff-443616b582d9-kube-api-access-szgg9\") pod \"maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:51.895371 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:51.895331 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:52.035100 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:52.035053 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482"] Jul 07 00:26:52.036996 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:26:52.036969 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85af738b_2226_4de6_a1ff_443616b582d9.slice/crio-837d2582774914504cce245429783392e9d0951fcd5a42aa1a8fa1c57e588f91 WatchSource:0}: Error finding container 837d2582774914504cce245429783392e9d0951fcd5a42aa1a8fa1c57e588f91: Status 404 returned error can't find the container with id 837d2582774914504cce245429783392e9d0951fcd5a42aa1a8fa1c57e588f91 Jul 07 00:26:52.092289 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:52.092249 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" event={"ID":"85af738b-2226-4de6-a1ff-443616b582d9","Type":"ContainerStarted","Data":"837d2582774914504cce245429783392e9d0951fcd5a42aa1a8fa1c57e588f91"} Jul 07 00:26:53.098625 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:53.098536 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" event={"ID":"85af738b-2226-4de6-a1ff-443616b582d9","Type":"ContainerStarted","Data":"cff920fb33984cb81da79ac4788a291c3b473aeab9db902069b589c78812ee15"} Jul 07 00:26:53.098985 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:53.098633 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:26:53.118497 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:53.118442 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" podStartSLOduration=1.434106454 podStartE2EDuration="2.118422237s" totalImagesPullingTime="684.315783ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:26:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:26:53.116765883 +0000 UTC m=+1779.089250546" watchObservedRunningTime="2026-07-07 00:26:53.118422237 +0000 UTC m=+1779.090906897" Jul 07 00:26:59.108716 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:26:59.108686 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:27:39.083509 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.083469 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482"] Jul 07 00:27:39.084084 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.083707 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" podUID="85af738b-2226-4de6-a1ff-443616b582d9" containerName="maas-api" containerID="cri-o://cff920fb33984cb81da79ac4788a291c3b473aeab9db902069b589c78812ee15" gracePeriod=30 Jul 07 00:27:39.104672 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.104632 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" podUID="85af738b-2226-4de6-a1ff-443616b582d9" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.46:8443/health\": dial tcp 10.134.0.46:8443: connect: connection refused" Jul 07 00:27:39.332445 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.332415 2562 generic.go:356] "Generic (PLEG): container finished" podID="85af738b-2226-4de6-a1ff-443616b582d9" containerID="cff920fb33984cb81da79ac4788a291c3b473aeab9db902069b589c78812ee15" exitCode=0 Jul 07 00:27:39.332598 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.332487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" event={"ID":"85af738b-2226-4de6-a1ff-443616b582d9","Type":"ContainerDied","Data":"cff920fb33984cb81da79ac4788a291c3b473aeab9db902069b589c78812ee15"} Jul 07 00:27:39.332598 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.332537 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" event={"ID":"85af738b-2226-4de6-a1ff-443616b582d9","Type":"ContainerDied","Data":"837d2582774914504cce245429783392e9d0951fcd5a42aa1a8fa1c57e588f91"} Jul 07 00:27:39.332598 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.332554 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="837d2582774914504cce245429783392e9d0951fcd5a42aa1a8fa1c57e588f91" Jul 07 00:27:39.337861 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.337804 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:27:39.395464 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.395432 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85af738b-2226-4de6-a1ff-443616b582d9-maas-api-tls\") pod \"85af738b-2226-4de6-a1ff-443616b582d9\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " Jul 07 00:27:39.395655 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.395527 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szgg9\" (UniqueName: \"kubernetes.io/projected/85af738b-2226-4de6-a1ff-443616b582d9-kube-api-access-szgg9\") pod \"85af738b-2226-4de6-a1ff-443616b582d9\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " Jul 07 00:27:39.395655 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.395551 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85af738b-2226-4de6-a1ff-443616b582d9-openshift-service-ca\") pod \"85af738b-2226-4de6-a1ff-443616b582d9\" (UID: \"85af738b-2226-4de6-a1ff-443616b582d9\") " Jul 07 00:27:39.395988 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.395960 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85af738b-2226-4de6-a1ff-443616b582d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "85af738b-2226-4de6-a1ff-443616b582d9" (UID: "85af738b-2226-4de6-a1ff-443616b582d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:27:39.397563 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.397540 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85af738b-2226-4de6-a1ff-443616b582d9-kube-api-access-szgg9" (OuterVolumeSpecName: "kube-api-access-szgg9") pod "85af738b-2226-4de6-a1ff-443616b582d9" (UID: "85af738b-2226-4de6-a1ff-443616b582d9"). InnerVolumeSpecName "kube-api-access-szgg9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:27:39.397643 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.397604 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85af738b-2226-4de6-a1ff-443616b582d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "85af738b-2226-4de6-a1ff-443616b582d9" (UID: "85af738b-2226-4de6-a1ff-443616b582d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:27:39.497019 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.496977 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szgg9\" (UniqueName: \"kubernetes.io/projected/85af738b-2226-4de6-a1ff-443616b582d9-kube-api-access-szgg9\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:27:39.497019 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.497009 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85af738b-2226-4de6-a1ff-443616b582d9-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:27:39.497019 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:39.497019 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85af738b-2226-4de6-a1ff-443616b582d9-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:27:40.338139 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:40.338109 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482" Jul 07 00:27:40.362449 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:40.362412 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482"] Jul 07 00:27:40.368355 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:40.368316 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-efed8c96-7c7f65b74d-lv482"] Jul 07 00:27:40.582969 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:27:40.582935 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85af738b-2226-4de6-a1ff-443616b582d9" path="/var/lib/kubelet/pods/85af738b-2226-4de6-a1ff-443616b582d9/volumes" Jul 07 00:29:32.546135 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.546098 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9"] Jul 07 00:29:32.546614 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.546435 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85af738b-2226-4de6-a1ff-443616b582d9" containerName="maas-api" Jul 07 00:29:32.546614 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.546445 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="85af738b-2226-4de6-a1ff-443616b582d9" containerName="maas-api" Jul 07 00:29:32.546614 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.546513 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="85af738b-2226-4de6-a1ff-443616b582d9" containerName="maas-api" Jul 07 00:29:32.549426 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.549404 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:32.553275 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.553250 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e9266c-serving-cert\"" Jul 07 00:29:32.565566 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.565538 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9"] Jul 07 00:29:32.613325 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.613280 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wzrq\" (UniqueName: \"kubernetes.io/projected/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-kube-api-access-9wzrq\") pod \"maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:32.613509 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.613365 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:32.613509 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.613392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-maas-api-tls\") pod \"maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:32.714601 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.714554 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:32.714601 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.714607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-maas-api-tls\") pod \"maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:32.714928 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.714691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wzrq\" (UniqueName: \"kubernetes.io/projected/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-kube-api-access-9wzrq\") pod \"maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:32.714928 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:29:32.714806 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-serving-cert: secret "maas-api-e2e-inf-a-e9266c-serving-cert" not found Jul 07 00:29:32.714928 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:29:32.714883 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-maas-api-tls podName:d2a7ffcc-a8da-45bc-a94d-2ce39b694342 nodeName:}" failed. No retries permitted until 2026-07-07 00:29:33.21486267 +0000 UTC m=+1939.187347322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-maas-api-tls") pod "maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" (UID: "d2a7ffcc-a8da-45bc-a94d-2ce39b694342") : secret "maas-api-e2e-inf-a-e9266c-serving-cert" not found Jul 07 00:29:32.715236 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.715219 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:32.725664 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:32.725633 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wzrq\" (UniqueName: \"kubernetes.io/projected/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-kube-api-access-9wzrq\") pod \"maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:33.220038 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:33.219998 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-maas-api-tls\") pod \"maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:33.222482 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:33.222452 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-maas-api-tls\") pod \"maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:33.460312 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:33.460269 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:33.613321 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:33.613295 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9"] Jul 07 00:29:33.615688 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:29:33.615662 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2a7ffcc_a8da_45bc_a94d_2ce39b694342.slice/crio-934dbf4ab0636c2f0543736d399cd9cc5b5a9b08a5cfaeac27e10aff83d6fe04 WatchSource:0}: Error finding container 934dbf4ab0636c2f0543736d399cd9cc5b5a9b08a5cfaeac27e10aff83d6fe04: Status 404 returned error can't find the container with id 934dbf4ab0636c2f0543736d399cd9cc5b5a9b08a5cfaeac27e10aff83d6fe04 Jul 07 00:29:33.617215 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:33.617194 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 00:29:33.911216 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:33.911177 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" event={"ID":"d2a7ffcc-a8da-45bc-a94d-2ce39b694342","Type":"ContainerStarted","Data":"934dbf4ab0636c2f0543736d399cd9cc5b5a9b08a5cfaeac27e10aff83d6fe04"} Jul 07 00:29:34.917159 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:34.917122 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" event={"ID":"d2a7ffcc-a8da-45bc-a94d-2ce39b694342","Type":"ContainerStarted","Data":"2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f"} Jul 07 00:29:34.917547 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:34.917305 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:34.957376 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:34.957323 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" podStartSLOduration=2.4519679 podStartE2EDuration="2.957307987s" totalImagesPullingTime="505.340087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:29:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:29:34.955319576 +0000 UTC m=+1940.927804237" watchObservedRunningTime="2026-07-07 00:29:34.957307987 +0000 UTC m=+1940.929792648" Jul 07 00:29:40.434629 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.434589 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6"] Jul 07 00:29:40.438734 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.438707 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:40.441319 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.441264 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-dd30cd-serving-cert\"" Jul 07 00:29:40.448897 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.448874 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6"] Jul 07 00:29:40.476933 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.476900 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6ff572-c85a-4380-bf81-0435300f9130-openshift-service-ca\") pod \"maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:40.477136 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.476953 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgc5v\" (UniqueName: \"kubernetes.io/projected/ea6ff572-c85a-4380-bf81-0435300f9130-kube-api-access-mgc5v\") pod \"maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:40.477136 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.476978 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea6ff572-c85a-4380-bf81-0435300f9130-maas-api-tls\") pod \"maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:40.577976 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.577940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgc5v\" (UniqueName: \"kubernetes.io/projected/ea6ff572-c85a-4380-bf81-0435300f9130-kube-api-access-mgc5v\") pod \"maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:40.577976 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.577990 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea6ff572-c85a-4380-bf81-0435300f9130-maas-api-tls\") pod \"maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:40.578223 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.578108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6ff572-c85a-4380-bf81-0435300f9130-openshift-service-ca\") pod \"maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:40.578223 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:29:40.578165 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-serving-cert: secret "maas-api-e2e-inf-b-dd30cd-serving-cert" not found Jul 07 00:29:40.578313 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:29:40.578261 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea6ff572-c85a-4380-bf81-0435300f9130-maas-api-tls podName:ea6ff572-c85a-4380-bf81-0435300f9130 nodeName:}" failed. No retries permitted until 2026-07-07 00:29:41.078239298 +0000 UTC m=+1947.050723954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea6ff572-c85a-4380-bf81-0435300f9130-maas-api-tls") pod "maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" (UID: "ea6ff572-c85a-4380-bf81-0435300f9130") : secret "maas-api-e2e-inf-b-dd30cd-serving-cert" not found Jul 07 00:29:40.578928 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.578903 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6ff572-c85a-4380-bf81-0435300f9130-openshift-service-ca\") pod \"maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:40.593967 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.593935 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgc5v\" (UniqueName: \"kubernetes.io/projected/ea6ff572-c85a-4380-bf81-0435300f9130-kube-api-access-mgc5v\") pod \"maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:40.928480 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:40.928452 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:29:41.082907 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:41.082868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea6ff572-c85a-4380-bf81-0435300f9130-maas-api-tls\") pod \"maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:41.085400 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:41.085373 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea6ff572-c85a-4380-bf81-0435300f9130-maas-api-tls\") pod \"maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:41.353546 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:41.353458 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:41.488129 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:41.488104 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6"] Jul 07 00:29:41.490318 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:29:41.490274 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6ff572_c85a_4380_bf81_0435300f9130.slice/crio-6d0c71e2d7749c50df0664a1f3a0a7547efbb427f14f0a893077cc6c6e56a53a WatchSource:0}: Error finding container 6d0c71e2d7749c50df0664a1f3a0a7547efbb427f14f0a893077cc6c6e56a53a: Status 404 returned error can't find the container with id 6d0c71e2d7749c50df0664a1f3a0a7547efbb427f14f0a893077cc6c6e56a53a Jul 07 00:29:41.953837 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:41.953802 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" event={"ID":"ea6ff572-c85a-4380-bf81-0435300f9130","Type":"ContainerStarted","Data":"f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132"} Jul 07 00:29:41.954000 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:41.953861 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" event={"ID":"ea6ff572-c85a-4380-bf81-0435300f9130","Type":"ContainerStarted","Data":"6d0c71e2d7749c50df0664a1f3a0a7547efbb427f14f0a893077cc6c6e56a53a"} Jul 07 00:29:41.954000 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:41.953924 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:29:41.976108 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:41.976043 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" podStartSLOduration=1.5935896999999999 podStartE2EDuration="1.97602598s" totalImagesPullingTime="382.43628ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:29:41.973485011 +0000 UTC m=+1947.945969674" watchObservedRunningTime="2026-07-07 00:29:41.97602598 +0000 UTC m=+1947.948510683" Jul 07 00:29:47.963850 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:29:47.963813 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:31:11.232843 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.232805 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9"] Jul 07 00:31:11.421988 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.421942 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" podUID="d2a7ffcc-a8da-45bc-a94d-2ce39b694342" containerName="maas-api" containerID="cri-o://2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f" gracePeriod=30 Jul 07 00:31:11.675020 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.674989 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:31:11.709960 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.709926 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-openshift-service-ca\") pod \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " Jul 07 00:31:11.710166 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.709965 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-maas-api-tls\") pod \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " Jul 07 00:31:11.710166 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.710087 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wzrq\" (UniqueName: \"kubernetes.io/projected/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-kube-api-access-9wzrq\") pod \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\" (UID: \"d2a7ffcc-a8da-45bc-a94d-2ce39b694342\") " Jul 07 00:31:11.710475 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.710387 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d2a7ffcc-a8da-45bc-a94d-2ce39b694342" (UID: "d2a7ffcc-a8da-45bc-a94d-2ce39b694342"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:31:11.712266 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.712240 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-kube-api-access-9wzrq" (OuterVolumeSpecName: "kube-api-access-9wzrq") pod "d2a7ffcc-a8da-45bc-a94d-2ce39b694342" (UID: "d2a7ffcc-a8da-45bc-a94d-2ce39b694342"). InnerVolumeSpecName "kube-api-access-9wzrq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:31:11.712401 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.712264 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d2a7ffcc-a8da-45bc-a94d-2ce39b694342" (UID: "d2a7ffcc-a8da-45bc-a94d-2ce39b694342"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:31:11.811178 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.811026 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wzrq\" (UniqueName: \"kubernetes.io/projected/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-kube-api-access-9wzrq\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:31:11.811178 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.811112 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:31:11.811178 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:11.811123 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2a7ffcc-a8da-45bc-a94d-2ce39b694342-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:31:12.428534 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:12.428497 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2a7ffcc-a8da-45bc-a94d-2ce39b694342" containerID="2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f" exitCode=0 Jul 07 00:31:12.428983 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:12.428547 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" Jul 07 00:31:12.428983 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:12.428588 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" event={"ID":"d2a7ffcc-a8da-45bc-a94d-2ce39b694342","Type":"ContainerDied","Data":"2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f"} Jul 07 00:31:12.428983 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:12.428646 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9" event={"ID":"d2a7ffcc-a8da-45bc-a94d-2ce39b694342","Type":"ContainerDied","Data":"934dbf4ab0636c2f0543736d399cd9cc5b5a9b08a5cfaeac27e10aff83d6fe04"} Jul 07 00:31:12.428983 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:12.428677 2562 scope.go:117] "RemoveContainer" containerID="2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f" Jul 07 00:31:12.439770 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:12.439747 2562 scope.go:117] "RemoveContainer" containerID="2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f" Jul 07 00:31:12.440172 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:31:12.440145 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f\": container with ID starting with 2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f not found: ID does not exist" containerID="2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f" Jul 07 00:31:12.440252 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:12.440184 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f"} err="failed to get container status \"2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f\": rpc error: code = NotFound desc = could not find container \"2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f\": container with ID starting with 2019e0fd43d915553669534f7f0b722395e2876402ca47d49082c0ff1496545f not found: ID does not exist" Jul 07 00:31:12.453202 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:12.453167 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9"] Jul 07 00:31:12.458452 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:12.458423 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9266c-5b6c67d8cf-mq8b9"] Jul 07 00:31:12.587861 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:12.587813 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2a7ffcc-a8da-45bc-a94d-2ce39b694342" path="/var/lib/kubelet/pods/d2a7ffcc-a8da-45bc-a94d-2ce39b694342/volumes" Jul 07 00:31:54.533438 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.533351 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6"] Jul 07 00:31:54.534007 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.533645 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" podUID="ea6ff572-c85a-4380-bf81-0435300f9130" containerName="maas-api" containerID="cri-o://f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132" gracePeriod=30 Jul 07 00:31:54.777022 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.776995 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:31:54.898604 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.898569 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6ff572-c85a-4380-bf81-0435300f9130-openshift-service-ca\") pod \"ea6ff572-c85a-4380-bf81-0435300f9130\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " Jul 07 00:31:54.898786 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.898669 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgc5v\" (UniqueName: \"kubernetes.io/projected/ea6ff572-c85a-4380-bf81-0435300f9130-kube-api-access-mgc5v\") pod \"ea6ff572-c85a-4380-bf81-0435300f9130\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " Jul 07 00:31:54.898786 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.898711 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea6ff572-c85a-4380-bf81-0435300f9130-maas-api-tls\") pod \"ea6ff572-c85a-4380-bf81-0435300f9130\" (UID: \"ea6ff572-c85a-4380-bf81-0435300f9130\") " Jul 07 00:31:54.899055 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.899027 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6ff572-c85a-4380-bf81-0435300f9130-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea6ff572-c85a-4380-bf81-0435300f9130" (UID: "ea6ff572-c85a-4380-bf81-0435300f9130"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:31:54.900833 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.900810 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6ff572-c85a-4380-bf81-0435300f9130-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea6ff572-c85a-4380-bf81-0435300f9130" (UID: "ea6ff572-c85a-4380-bf81-0435300f9130"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:31:54.900890 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.900816 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea6ff572-c85a-4380-bf81-0435300f9130-kube-api-access-mgc5v" (OuterVolumeSpecName: "kube-api-access-mgc5v") pod "ea6ff572-c85a-4380-bf81-0435300f9130" (UID: "ea6ff572-c85a-4380-bf81-0435300f9130"). InnerVolumeSpecName "kube-api-access-mgc5v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:31:54.999555 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.999516 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgc5v\" (UniqueName: \"kubernetes.io/projected/ea6ff572-c85a-4380-bf81-0435300f9130-kube-api-access-mgc5v\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:31:54.999555 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.999547 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea6ff572-c85a-4380-bf81-0435300f9130-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:31:54.999555 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:54.999556 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6ff572-c85a-4380-bf81-0435300f9130-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:31:55.647326 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:55.647283 2562 generic.go:356] "Generic (PLEG): container finished" podID="ea6ff572-c85a-4380-bf81-0435300f9130" containerID="f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132" exitCode=0 Jul 07 00:31:55.647326 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:55.647324 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" Jul 07 00:31:55.647816 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:55.647361 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" event={"ID":"ea6ff572-c85a-4380-bf81-0435300f9130","Type":"ContainerDied","Data":"f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132"} Jul 07 00:31:55.647816 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:55.647430 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6" event={"ID":"ea6ff572-c85a-4380-bf81-0435300f9130","Type":"ContainerDied","Data":"6d0c71e2d7749c50df0664a1f3a0a7547efbb427f14f0a893077cc6c6e56a53a"} Jul 07 00:31:55.647816 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:55.647460 2562 scope.go:117] "RemoveContainer" containerID="f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132" Jul 07 00:31:55.657450 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:55.657432 2562 scope.go:117] "RemoveContainer" containerID="f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132" Jul 07 00:31:55.657750 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:31:55.657731 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132\": container with ID starting with f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132 not found: ID does not exist" containerID="f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132" Jul 07 00:31:55.657797 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:55.657761 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132"} err="failed to get container status \"f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132\": rpc error: code = NotFound desc = could not find container \"f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132\": container with ID starting with f65929bc12bdd0e302f50f9ac42723e09812112ca1cd730c02a4c5f5703f8132 not found: ID does not exist" Jul 07 00:31:55.672653 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:55.672620 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6"] Jul 07 00:31:55.676650 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:55.676622 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dd30cd-67b4f7bf6f-fjjn6"] Jul 07 00:31:56.582932 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:31:56.582898 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea6ff572-c85a-4380-bf81-0435300f9130" path="/var/lib/kubelet/pods/ea6ff572-c85a-4380-bf81-0435300f9130/volumes" Jul 07 00:32:04.249279 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.249238 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l"] Jul 07 00:32:04.249767 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.249679 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a7ffcc-a8da-45bc-a94d-2ce39b694342" containerName="maas-api" Jul 07 00:32:04.249767 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.249698 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a7ffcc-a8da-45bc-a94d-2ce39b694342" containerName="maas-api" Jul 07 00:32:04.249767 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.249728 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6ff572-c85a-4380-bf81-0435300f9130" containerName="maas-api" Jul 07 00:32:04.249767 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.249735 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6ff572-c85a-4380-bf81-0435300f9130" containerName="maas-api" Jul 07 00:32:04.249990 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.249825 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6ff572-c85a-4380-bf81-0435300f9130" containerName="maas-api" Jul 07 00:32:04.249990 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.249840 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2a7ffcc-a8da-45bc-a94d-2ce39b694342" containerName="maas-api" Jul 07 00:32:04.254124 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.254105 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:04.256766 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.256747 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-259681-serving-cert\"" Jul 07 00:32:04.265527 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.265500 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l"] Jul 07 00:32:04.269199 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.269177 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mprl4\" (UniqueName: \"kubernetes.io/projected/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-kube-api-access-mprl4\") pod \"maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:04.269305 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.269237 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-maas-api-tls\") pod \"maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:04.269379 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.269358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-openshift-service-ca\") pod \"maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:04.370409 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.370371 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-maas-api-tls\") pod \"maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:04.370612 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.370463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-openshift-service-ca\") pod \"maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:04.370612 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.370492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprl4\" (UniqueName: \"kubernetes.io/projected/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-kube-api-access-mprl4\") pod \"maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:04.370612 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:32:04.370511 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-259681-serving-cert: secret "maas-api-e2e-api-a-259681-serving-cert" not found Jul 07 00:32:04.370612 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:32:04.370579 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-maas-api-tls podName:6ce4016f-c66d-45b1-8988-f2e3ac3b5ada nodeName:}" failed. No retries permitted until 2026-07-07 00:32:04.870562577 +0000 UTC m=+2090.843047216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-maas-api-tls") pod "maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" (UID: "6ce4016f-c66d-45b1-8988-f2e3ac3b5ada") : secret "maas-api-e2e-api-a-259681-serving-cert" not found Jul 07 00:32:04.371249 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.371229 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-openshift-service-ca\") pod \"maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:04.380079 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.380038 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mprl4\" (UniqueName: \"kubernetes.io/projected/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-kube-api-access-mprl4\") pod \"maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:04.874156 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.874122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-maas-api-tls\") pod \"maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:04.876599 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:04.876576 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-maas-api-tls\") pod \"maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:05.167740 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:05.167661 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:05.314644 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:05.314617 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l"] Jul 07 00:32:05.316655 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:32:05.316624 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ce4016f_c66d_45b1_8988_f2e3ac3b5ada.slice/crio-472723b27762540875e2f2803532e3b0962da951c72a58fd530259386589469e WatchSource:0}: Error finding container 472723b27762540875e2f2803532e3b0962da951c72a58fd530259386589469e: Status 404 returned error can't find the container with id 472723b27762540875e2f2803532e3b0962da951c72a58fd530259386589469e Jul 07 00:32:05.699188 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:05.699150 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" event={"ID":"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada","Type":"ContainerStarted","Data":"472723b27762540875e2f2803532e3b0962da951c72a58fd530259386589469e"} Jul 07 00:32:06.705535 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:06.705489 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" event={"ID":"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada","Type":"ContainerStarted","Data":"9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb"} Jul 07 00:32:06.705951 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:06.705573 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:06.722608 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:06.722557 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" podStartSLOduration=2.196003768 podStartE2EDuration="2.722541389s" totalImagesPullingTime="526.537621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:32:06.721488908 +0000 UTC m=+2092.693973571" watchObservedRunningTime="2026-07-07 00:32:06.722541389 +0000 UTC m=+2092.695026049" Jul 07 00:32:12.427666 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.427615 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f"] Jul 07 00:32:12.431954 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.431928 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:12.435241 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.435218 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-26d2ae-serving-cert\"" Jul 07 00:32:12.440614 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.440580 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f"] Jul 07 00:32:12.544292 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.544250 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebfd09a-f0ff-4050-be51-17388b2ebd12-openshift-service-ca\") pod \"maas-api-e2e-api-b-26d2ae-d74df966-p7h2f\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:12.544461 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.544376 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebfd09a-f0ff-4050-be51-17388b2ebd12-maas-api-tls\") pod \"maas-api-e2e-api-b-26d2ae-d74df966-p7h2f\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:12.544461 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.544448 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnmwf\" (UniqueName: \"kubernetes.io/projected/2ebfd09a-f0ff-4050-be51-17388b2ebd12-kube-api-access-rnmwf\") pod \"maas-api-e2e-api-b-26d2ae-d74df966-p7h2f\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:12.645433 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.645392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebfd09a-f0ff-4050-be51-17388b2ebd12-openshift-service-ca\") pod \"maas-api-e2e-api-b-26d2ae-d74df966-p7h2f\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:12.645668 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.645473 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebfd09a-f0ff-4050-be51-17388b2ebd12-maas-api-tls\") pod \"maas-api-e2e-api-b-26d2ae-d74df966-p7h2f\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:12.645668 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.645521 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnmwf\" (UniqueName: \"kubernetes.io/projected/2ebfd09a-f0ff-4050-be51-17388b2ebd12-kube-api-access-rnmwf\") pod \"maas-api-e2e-api-b-26d2ae-d74df966-p7h2f\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:12.646137 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.646111 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebfd09a-f0ff-4050-be51-17388b2ebd12-openshift-service-ca\") pod \"maas-api-e2e-api-b-26d2ae-d74df966-p7h2f\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:12.648118 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.648089 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebfd09a-f0ff-4050-be51-17388b2ebd12-maas-api-tls\") pod \"maas-api-e2e-api-b-26d2ae-d74df966-p7h2f\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:12.676126 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.676060 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnmwf\" (UniqueName: \"kubernetes.io/projected/2ebfd09a-f0ff-4050-be51-17388b2ebd12-kube-api-access-rnmwf\") pod \"maas-api-e2e-api-b-26d2ae-d74df966-p7h2f\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:12.717098 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.716992 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:12.746452 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.746423 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:12.939499 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:12.939463 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f"] Jul 07 00:32:12.942060 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:32:12.942019 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ebfd09a_f0ff_4050_be51_17388b2ebd12.slice/crio-54e5b771a406896d081efec395e9618050b82e980cae6260382a45e93a62089c WatchSource:0}: Error finding container 54e5b771a406896d081efec395e9618050b82e980cae6260382a45e93a62089c: Status 404 returned error can't find the container with id 54e5b771a406896d081efec395e9618050b82e980cae6260382a45e93a62089c Jul 07 00:32:13.746441 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:13.746033 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" event={"ID":"2ebfd09a-f0ff-4050-be51-17388b2ebd12","Type":"ContainerStarted","Data":"215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026"} Jul 07 00:32:13.746441 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:13.746143 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" event={"ID":"2ebfd09a-f0ff-4050-be51-17388b2ebd12","Type":"ContainerStarted","Data":"54e5b771a406896d081efec395e9618050b82e980cae6260382a45e93a62089c"} Jul 07 00:32:13.746441 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:13.746188 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:13.765794 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:13.765740 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" podStartSLOduration=1.422956776 podStartE2EDuration="1.765721113s" totalImagesPullingTime="342.764337ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:32:13.763290502 +0000 UTC m=+2099.735775164" watchObservedRunningTime="2026-07-07 00:32:13.765721113 +0000 UTC m=+2099.738205792" Jul 07 00:32:19.757499 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:19.757463 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:25.794957 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:25.794919 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l"] Jul 07 00:32:25.795617 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:25.795185 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" podUID="6ce4016f-c66d-45b1-8988-f2e3ac3b5ada" containerName="maas-api" containerID="cri-o://9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb" gracePeriod=30 Jul 07 00:32:26.043654 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.043623 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:26.178403 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.178370 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-maas-api-tls\") pod \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " Jul 07 00:32:26.178642 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.178438 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mprl4\" (UniqueName: \"kubernetes.io/projected/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-kube-api-access-mprl4\") pod \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " Jul 07 00:32:26.178642 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.178472 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-openshift-service-ca\") pod \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\" (UID: \"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada\") " Jul 07 00:32:26.178903 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.178873 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6ce4016f-c66d-45b1-8988-f2e3ac3b5ada" (UID: "6ce4016f-c66d-45b1-8988-f2e3ac3b5ada"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:32:26.180670 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.180645 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6ce4016f-c66d-45b1-8988-f2e3ac3b5ada" (UID: "6ce4016f-c66d-45b1-8988-f2e3ac3b5ada"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:32:26.180747 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.180651 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-kube-api-access-mprl4" (OuterVolumeSpecName: "kube-api-access-mprl4") pod "6ce4016f-c66d-45b1-8988-f2e3ac3b5ada" (UID: "6ce4016f-c66d-45b1-8988-f2e3ac3b5ada"). InnerVolumeSpecName "kube-api-access-mprl4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:32:26.280154 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.280099 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:32:26.280154 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.280149 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mprl4\" (UniqueName: \"kubernetes.io/projected/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-kube-api-access-mprl4\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:32:26.280154 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.280159 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:32:26.813935 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.813890 2562 generic.go:356] "Generic (PLEG): container finished" podID="6ce4016f-c66d-45b1-8988-f2e3ac3b5ada" containerID="9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb" exitCode=0 Jul 07 00:32:26.813935 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.813940 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" Jul 07 00:32:26.814468 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.813976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" event={"ID":"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada","Type":"ContainerDied","Data":"9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb"} Jul 07 00:32:26.814468 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.814035 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l" event={"ID":"6ce4016f-c66d-45b1-8988-f2e3ac3b5ada","Type":"ContainerDied","Data":"472723b27762540875e2f2803532e3b0962da951c72a58fd530259386589469e"} Jul 07 00:32:26.814468 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.814057 2562 scope.go:117] "RemoveContainer" containerID="9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb" Jul 07 00:32:26.823106 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.823083 2562 scope.go:117] "RemoveContainer" containerID="9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb" Jul 07 00:32:26.823408 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:32:26.823389 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb\": container with ID starting with 9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb not found: ID does not exist" containerID="9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb" Jul 07 00:32:26.823468 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.823418 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb"} err="failed to get container status \"9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb\": rpc error: code = NotFound desc = could not find container \"9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb\": container with ID starting with 9bb76df5ac861fbede8f3980f18781bf8434d86a1979a5f08a995de1796b6abb not found: ID does not exist" Jul 07 00:32:26.833127 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.833098 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l"] Jul 07 00:32:26.837556 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:26.837526 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-259681-66fd7b8d5b-m6s9l"] Jul 07 00:32:28.582784 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:28.582751 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ce4016f-c66d-45b1-8988-f2e3ac3b5ada" path="/var/lib/kubelet/pods/6ce4016f-c66d-45b1-8988-f2e3ac3b5ada/volumes" Jul 07 00:32:32.542645 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.542597 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f"] Jul 07 00:32:32.543051 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.542895 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" podUID="2ebfd09a-f0ff-4050-be51-17388b2ebd12" containerName="maas-api" containerID="cri-o://215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026" gracePeriod=30 Jul 07 00:32:32.791682 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.791654 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:32.847917 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.847813 2562 generic.go:356] "Generic (PLEG): container finished" podID="2ebfd09a-f0ff-4050-be51-17388b2ebd12" containerID="215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026" exitCode=0 Jul 07 00:32:32.847917 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.847871 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" Jul 07 00:32:32.847917 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.847896 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" event={"ID":"2ebfd09a-f0ff-4050-be51-17388b2ebd12","Type":"ContainerDied","Data":"215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026"} Jul 07 00:32:32.848236 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.847952 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f" event={"ID":"2ebfd09a-f0ff-4050-be51-17388b2ebd12","Type":"ContainerDied","Data":"54e5b771a406896d081efec395e9618050b82e980cae6260382a45e93a62089c"} Jul 07 00:32:32.848236 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.847975 2562 scope.go:117] "RemoveContainer" containerID="215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026" Jul 07 00:32:32.856779 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.856750 2562 scope.go:117] "RemoveContainer" containerID="215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026" Jul 07 00:32:32.857052 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:32:32.857032 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026\": container with ID starting with 215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026 not found: ID does not exist" containerID="215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026" Jul 07 00:32:32.857145 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.857059 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026"} err="failed to get container status \"215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026\": rpc error: code = NotFound desc = could not find container \"215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026\": container with ID starting with 215022475a25fbb3ba6530abd5483ba3419fed85b17c4490cda7e38aded37026 not found: ID does not exist" Jul 07 00:32:32.941362 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.941324 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebfd09a-f0ff-4050-be51-17388b2ebd12-openshift-service-ca\") pod \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " Jul 07 00:32:32.941362 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.941367 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnmwf\" (UniqueName: \"kubernetes.io/projected/2ebfd09a-f0ff-4050-be51-17388b2ebd12-kube-api-access-rnmwf\") pod \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " Jul 07 00:32:32.941591 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.941413 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebfd09a-f0ff-4050-be51-17388b2ebd12-maas-api-tls\") pod \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\" (UID: \"2ebfd09a-f0ff-4050-be51-17388b2ebd12\") " Jul 07 00:32:32.941799 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.941773 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ebfd09a-f0ff-4050-be51-17388b2ebd12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2ebfd09a-f0ff-4050-be51-17388b2ebd12" (UID: "2ebfd09a-f0ff-4050-be51-17388b2ebd12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:32:32.943646 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.943614 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ebfd09a-f0ff-4050-be51-17388b2ebd12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2ebfd09a-f0ff-4050-be51-17388b2ebd12" (UID: "2ebfd09a-f0ff-4050-be51-17388b2ebd12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:32:32.943646 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:32.943644 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ebfd09a-f0ff-4050-be51-17388b2ebd12-kube-api-access-rnmwf" (OuterVolumeSpecName: "kube-api-access-rnmwf") pod "2ebfd09a-f0ff-4050-be51-17388b2ebd12" (UID: "2ebfd09a-f0ff-4050-be51-17388b2ebd12"). InnerVolumeSpecName "kube-api-access-rnmwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:32:33.042241 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:33.042208 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebfd09a-f0ff-4050-be51-17388b2ebd12-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:32:33.042241 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:33.042234 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebfd09a-f0ff-4050-be51-17388b2ebd12-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:32:33.042241 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:33.042246 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnmwf\" (UniqueName: \"kubernetes.io/projected/2ebfd09a-f0ff-4050-be51-17388b2ebd12-kube-api-access-rnmwf\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:32:33.173332 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:33.173302 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f"] Jul 07 00:32:33.177465 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:33.177430 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-26d2ae-d74df966-p7h2f"] Jul 07 00:32:34.582876 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:34.582836 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ebfd09a-f0ff-4050-be51-17388b2ebd12" path="/var/lib/kubelet/pods/2ebfd09a-f0ff-4050-be51-17388b2ebd12/volumes" Jul 07 00:32:42.545803 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.545764 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv"] Jul 07 00:32:42.546226 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.546147 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ce4016f-c66d-45b1-8988-f2e3ac3b5ada" containerName="maas-api" Jul 07 00:32:42.546226 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.546161 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ce4016f-c66d-45b1-8988-f2e3ac3b5ada" containerName="maas-api" Jul 07 00:32:42.546226 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.546184 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ebfd09a-f0ff-4050-be51-17388b2ebd12" containerName="maas-api" Jul 07 00:32:42.546226 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.546190 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ebfd09a-f0ff-4050-be51-17388b2ebd12" containerName="maas-api" Jul 07 00:32:42.546362 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.546242 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ce4016f-c66d-45b1-8988-f2e3ac3b5ada" containerName="maas-api" Jul 07 00:32:42.546362 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.546250 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ebfd09a-f0ff-4050-be51-17388b2ebd12" containerName="maas-api" Jul 07 00:32:42.550716 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.550689 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:42.553694 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.553676 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-a8caec-serving-cert\"" Jul 07 00:32:42.571516 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.571479 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv"] Jul 07 00:32:42.624078 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.624027 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twrk5\" (UniqueName: \"kubernetes.io/projected/288cd1bf-89a4-4adb-8d82-26b1043a50f5-kube-api-access-twrk5\") pod \"maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:42.624254 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.624114 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/288cd1bf-89a4-4adb-8d82-26b1043a50f5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:42.624254 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.624163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288cd1bf-89a4-4adb-8d82-26b1043a50f5-maas-api-tls\") pod \"maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:42.725302 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.725259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/288cd1bf-89a4-4adb-8d82-26b1043a50f5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:42.725521 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.725333 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288cd1bf-89a4-4adb-8d82-26b1043a50f5-maas-api-tls\") pod \"maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:42.725521 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.725379 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twrk5\" (UniqueName: \"kubernetes.io/projected/288cd1bf-89a4-4adb-8d82-26b1043a50f5-kube-api-access-twrk5\") pod \"maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:42.725521 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:32:42.725500 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-serving-cert: secret "maas-api-e2e-shared-a-a8caec-serving-cert" not found Jul 07 00:32:42.725706 ip-10-0-130-52 kubenswrapper[2562]: E0707 00:32:42.725602 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/288cd1bf-89a4-4adb-8d82-26b1043a50f5-maas-api-tls podName:288cd1bf-89a4-4adb-8d82-26b1043a50f5 nodeName:}" failed. No retries permitted until 2026-07-07 00:32:43.225577854 +0000 UTC m=+2129.198062507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/288cd1bf-89a4-4adb-8d82-26b1043a50f5-maas-api-tls") pod "maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" (UID: "288cd1bf-89a4-4adb-8d82-26b1043a50f5") : secret "maas-api-e2e-shared-a-a8caec-serving-cert" not found Jul 07 00:32:42.725953 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.725934 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/288cd1bf-89a4-4adb-8d82-26b1043a50f5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:42.735310 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:42.735274 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twrk5\" (UniqueName: \"kubernetes.io/projected/288cd1bf-89a4-4adb-8d82-26b1043a50f5-kube-api-access-twrk5\") pod \"maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:43.230346 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:43.230299 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288cd1bf-89a4-4adb-8d82-26b1043a50f5-maas-api-tls\") pod \"maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:43.233018 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:43.232991 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288cd1bf-89a4-4adb-8d82-26b1043a50f5-maas-api-tls\") pod \"maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:43.468573 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:43.468531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:43.613692 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:43.613648 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv"] Jul 07 00:32:43.614589 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:32:43.614559 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod288cd1bf_89a4_4adb_8d82_26b1043a50f5.slice/crio-0228396d904ac4605fb662608d2d6af98666dcd5d3f8f58436c8b3d6d7ee9af4 WatchSource:0}: Error finding container 0228396d904ac4605fb662608d2d6af98666dcd5d3f8f58436c8b3d6d7ee9af4: Status 404 returned error can't find the container with id 0228396d904ac4605fb662608d2d6af98666dcd5d3f8f58436c8b3d6d7ee9af4 Jul 07 00:32:43.906423 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:43.906384 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" event={"ID":"288cd1bf-89a4-4adb-8d82-26b1043a50f5","Type":"ContainerStarted","Data":"0228396d904ac4605fb662608d2d6af98666dcd5d3f8f58436c8b3d6d7ee9af4"} Jul 07 00:32:44.913253 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:44.913219 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" event={"ID":"288cd1bf-89a4-4adb-8d82-26b1043a50f5","Type":"ContainerStarted","Data":"de5ebbda3804b453669ae6924221dc07842888fd297b0aa36cbe35c38750968f"} Jul 07 00:32:44.913737 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:44.913285 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:44.930817 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:44.930767 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" podStartSLOduration=2.472399828 podStartE2EDuration="2.93074939s" totalImagesPullingTime="458.349562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:32:44.929637812 +0000 UTC m=+2130.902122488" watchObservedRunningTime="2026-07-07 00:32:44.93074939 +0000 UTC m=+2130.903234050" Jul 07 00:32:50.631900 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.631850 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb"] Jul 07 00:32:50.635993 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.635958 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:50.638674 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.638645 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-3ee067-serving-cert\"" Jul 07 00:32:50.647965 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.647931 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb"] Jul 07 00:32:50.699945 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.699909 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-maas-api-tls\") pod \"maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:50.700187 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.699959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dzwk\" (UniqueName: \"kubernetes.io/projected/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-kube-api-access-8dzwk\") pod \"maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:50.700187 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.700138 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:50.800897 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.800859 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-maas-api-tls\") pod \"maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:50.801126 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.800905 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dzwk\" (UniqueName: \"kubernetes.io/projected/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-kube-api-access-8dzwk\") pod \"maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:50.801126 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.800986 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:50.801735 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.801711 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:50.803403 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.803379 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-maas-api-tls\") pod \"maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:50.813193 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.813164 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dzwk\" (UniqueName: \"kubernetes.io/projected/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-kube-api-access-8dzwk\") pod \"maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:50.924784 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.924702 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:32:50.950195 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:50.950155 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:51.099678 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:51.099649 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb"] Jul 07 00:32:51.101898 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:32:51.101863 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda694d420_c2dc_4f2e_8ee6_c9f96509adb6.slice/crio-23e8d777e045f6e5af8b8a21a973f82e1322e60aa0b6c7834f81065cc3981d08 WatchSource:0}: Error finding container 23e8d777e045f6e5af8b8a21a973f82e1322e60aa0b6c7834f81065cc3981d08: Status 404 returned error can't find the container with id 23e8d777e045f6e5af8b8a21a973f82e1322e60aa0b6c7834f81065cc3981d08 Jul 07 00:32:51.954421 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:51.954376 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" event={"ID":"a694d420-c2dc-4f2e-8ee6-c9f96509adb6","Type":"ContainerStarted","Data":"c271b7f7cebe30dc8fbfe9d6f2476981c05d2384c767f3b8f9a6d0e2d40a7bbc"} Jul 07 00:32:51.954850 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:51.954449 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" event={"ID":"a694d420-c2dc-4f2e-8ee6-c9f96509adb6","Type":"ContainerStarted","Data":"23e8d777e045f6e5af8b8a21a973f82e1322e60aa0b6c7834f81065cc3981d08"} Jul 07 00:32:51.954850 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:51.954491 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:32:51.972686 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:51.972625 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" podStartSLOduration=1.543181178 podStartE2EDuration="1.972606073s" totalImagesPullingTime="429.424895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:32:51.971776482 +0000 UTC m=+2137.944261179" watchObservedRunningTime="2026-07-07 00:32:51.972606073 +0000 UTC m=+2137.945090733" Jul 07 00:32:57.965623 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:32:57.965591 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:33:16.844745 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:33:16.844671 2562 scope.go:117] "RemoveContainer" containerID="cff920fb33984cb81da79ac4788a291c3b473aeab9db902069b589c78812ee15" Jul 07 00:37:56.253088 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.252980 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv"] Jul 07 00:37:56.253677 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.253258 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" podUID="288cd1bf-89a4-4adb-8d82-26b1043a50f5" containerName="maas-api" containerID="cri-o://de5ebbda3804b453669ae6924221dc07842888fd297b0aa36cbe35c38750968f" gracePeriod=30 Jul 07 00:37:56.507583 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.507510 2562 generic.go:356] "Generic (PLEG): container finished" podID="288cd1bf-89a4-4adb-8d82-26b1043a50f5" containerID="de5ebbda3804b453669ae6924221dc07842888fd297b0aa36cbe35c38750968f" exitCode=0 Jul 07 00:37:56.507702 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.507588 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" event={"ID":"288cd1bf-89a4-4adb-8d82-26b1043a50f5","Type":"ContainerDied","Data":"de5ebbda3804b453669ae6924221dc07842888fd297b0aa36cbe35c38750968f"} Jul 07 00:37:56.507702 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.507646 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" event={"ID":"288cd1bf-89a4-4adb-8d82-26b1043a50f5","Type":"ContainerDied","Data":"0228396d904ac4605fb662608d2d6af98666dcd5d3f8f58436c8b3d6d7ee9af4"} Jul 07 00:37:56.507702 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.507664 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0228396d904ac4605fb662608d2d6af98666dcd5d3f8f58436c8b3d6d7ee9af4" Jul 07 00:37:56.514009 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.513988 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:37:56.686745 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.686688 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288cd1bf-89a4-4adb-8d82-26b1043a50f5-maas-api-tls\") pod \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " Jul 07 00:37:56.686955 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.686799 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/288cd1bf-89a4-4adb-8d82-26b1043a50f5-openshift-service-ca\") pod \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " Jul 07 00:37:56.686955 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.686900 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twrk5\" (UniqueName: \"kubernetes.io/projected/288cd1bf-89a4-4adb-8d82-26b1043a50f5-kube-api-access-twrk5\") pod \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\" (UID: \"288cd1bf-89a4-4adb-8d82-26b1043a50f5\") " Jul 07 00:37:56.687250 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.687216 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/288cd1bf-89a4-4adb-8d82-26b1043a50f5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "288cd1bf-89a4-4adb-8d82-26b1043a50f5" (UID: "288cd1bf-89a4-4adb-8d82-26b1043a50f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:37:56.689470 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.689442 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/288cd1bf-89a4-4adb-8d82-26b1043a50f5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "288cd1bf-89a4-4adb-8d82-26b1043a50f5" (UID: "288cd1bf-89a4-4adb-8d82-26b1043a50f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:37:56.689691 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.689674 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/288cd1bf-89a4-4adb-8d82-26b1043a50f5-kube-api-access-twrk5" (OuterVolumeSpecName: "kube-api-access-twrk5") pod "288cd1bf-89a4-4adb-8d82-26b1043a50f5" (UID: "288cd1bf-89a4-4adb-8d82-26b1043a50f5"). InnerVolumeSpecName "kube-api-access-twrk5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:37:56.788573 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.788488 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twrk5\" (UniqueName: \"kubernetes.io/projected/288cd1bf-89a4-4adb-8d82-26b1043a50f5-kube-api-access-twrk5\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:37:56.788573 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.788514 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/288cd1bf-89a4-4adb-8d82-26b1043a50f5-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:37:56.788573 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:56.788524 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/288cd1bf-89a4-4adb-8d82-26b1043a50f5-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:37:57.512079 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:57.512044 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv" Jul 07 00:37:57.536438 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:57.536399 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv"] Jul 07 00:37:57.540200 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:57.540169 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a8caec-5456db4d85-sp7mv"] Jul 07 00:37:58.582781 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:37:58.582748 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="288cd1bf-89a4-4adb-8d82-26b1043a50f5" path="/var/lib/kubelet/pods/288cd1bf-89a4-4adb-8d82-26b1043a50f5/volumes" Jul 07 00:38:22.403270 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.403232 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb"] Jul 07 00:38:22.403737 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.403466 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" podUID="a694d420-c2dc-4f2e-8ee6-c9f96509adb6" containerName="maas-api" containerID="cri-o://c271b7f7cebe30dc8fbfe9d6f2476981c05d2384c767f3b8f9a6d0e2d40a7bbc" gracePeriod=30 Jul 07 00:38:22.663233 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.663150 2562 generic.go:356] "Generic (PLEG): container finished" podID="a694d420-c2dc-4f2e-8ee6-c9f96509adb6" containerID="c271b7f7cebe30dc8fbfe9d6f2476981c05d2384c767f3b8f9a6d0e2d40a7bbc" exitCode=0 Jul 07 00:38:22.663359 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.663243 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" event={"ID":"a694d420-c2dc-4f2e-8ee6-c9f96509adb6","Type":"ContainerDied","Data":"c271b7f7cebe30dc8fbfe9d6f2476981c05d2384c767f3b8f9a6d0e2d40a7bbc"} Jul 07 00:38:22.670223 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.670198 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:38:22.705322 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.705287 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-openshift-service-ca\") pod \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " Jul 07 00:38:22.705486 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.705344 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-maas-api-tls\") pod \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " Jul 07 00:38:22.705486 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.705421 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dzwk\" (UniqueName: \"kubernetes.io/projected/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-kube-api-access-8dzwk\") pod \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\" (UID: \"a694d420-c2dc-4f2e-8ee6-c9f96509adb6\") " Jul 07 00:38:22.705744 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.705713 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a694d420-c2dc-4f2e-8ee6-c9f96509adb6" (UID: "a694d420-c2dc-4f2e-8ee6-c9f96509adb6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:38:22.707797 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.707760 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a694d420-c2dc-4f2e-8ee6-c9f96509adb6" (UID: "a694d420-c2dc-4f2e-8ee6-c9f96509adb6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:38:22.707926 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.707874 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-kube-api-access-8dzwk" (OuterVolumeSpecName: "kube-api-access-8dzwk") pod "a694d420-c2dc-4f2e-8ee6-c9f96509adb6" (UID: "a694d420-c2dc-4f2e-8ee6-c9f96509adb6"). InnerVolumeSpecName "kube-api-access-8dzwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:38:22.806584 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.806529 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:38:22.806584 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.806579 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dzwk\" (UniqueName: \"kubernetes.io/projected/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-kube-api-access-8dzwk\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:38:22.806584 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:22.806591 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a694d420-c2dc-4f2e-8ee6-c9f96509adb6-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\"" Jul 07 00:38:23.669620 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:23.669585 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" Jul 07 00:38:23.670160 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:23.669623 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb" event={"ID":"a694d420-c2dc-4f2e-8ee6-c9f96509adb6","Type":"ContainerDied","Data":"23e8d777e045f6e5af8b8a21a973f82e1322e60aa0b6c7834f81065cc3981d08"} Jul 07 00:38:23.670160 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:23.669700 2562 scope.go:117] "RemoveContainer" containerID="c271b7f7cebe30dc8fbfe9d6f2476981c05d2384c767f3b8f9a6d0e2d40a7bbc" Jul 07 00:38:23.692639 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:23.692605 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb"] Jul 07 00:38:23.696030 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:23.696006 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3ee067-f86fb568f-rw6jb"] Jul 07 00:38:24.582905 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:24.582869 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a694d420-c2dc-4f2e-8ee6-c9f96509adb6" path="/var/lib/kubelet/pods/a694d420-c2dc-4f2e-8ee6-c9f96509adb6/volumes" Jul 07 00:38:34.417490 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:34.417448 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5868db4bd6-qvcsh_8623a11e-31f3-40aa-b164-3781d41596e0/manager/0.log" Jul 07 00:38:34.788921 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:34.788822 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b6d6b477c-4trmc_09ebb26f-ac7b-4a20-be81-608ac47a4038/manager/0.log" Jul 07 00:38:34.795691 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:34.795670 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b6d6b477c-4trmc_09ebb26f-ac7b-4a20-be81-608ac47a4038/copy-manifests/0.log" Jul 07 00:38:35.876638 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:35.876605 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c_4c2be1b3-a140-4e94-8981-b9f8d257846f/util/0.log" Jul 07 00:38:35.883751 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:35.883727 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c_4c2be1b3-a140-4e94-8981-b9f8d257846f/pull/0.log" Jul 07 00:38:35.890303 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:35.890265 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c_4c2be1b3-a140-4e94-8981-b9f8d257846f/extract/0.log" Jul 07 00:38:36.011364 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:36.011338 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6_d63a0916-a7af-4d2b-941d-242ee4704ba0/util/0.log" Jul 07 00:38:36.018481 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:36.018462 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6_d63a0916-a7af-4d2b-941d-242ee4704ba0/pull/0.log" Jul 07 00:38:36.024944 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:36.024925 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6_d63a0916-a7af-4d2b-941d-242ee4704ba0/extract/0.log" Jul 07 00:38:36.146460 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:36.146376 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf_ed65c1a6-9419-47ac-ba01-8d7dc007b18f/pull/0.log" Jul 07 00:38:36.153511 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:36.153489 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf_ed65c1a6-9419-47ac-ba01-8d7dc007b18f/extract/0.log" Jul 07 00:38:36.159982 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:36.159961 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf_ed65c1a6-9419-47ac-ba01-8d7dc007b18f/util/0.log" Jul 07 00:38:36.284369 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:36.284342 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz_4c1fb982-b6f5-4687-98c5-e6207076c890/util/0.log" Jul 07 00:38:36.292140 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:36.292118 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz_4c1fb982-b6f5-4687-98c5-e6207076c890/pull/0.log" Jul 07 00:38:36.299430 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:36.299409 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz_4c1fb982-b6f5-4687-98c5-e6207076c890/extract/0.log" Jul 07 00:38:37.826289 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:37.826254 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-59f7c_2b56aae5-875d-4c1f-82e7-70e360f432b3/discovery/0.log" Jul 07 00:38:38.306912 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:38.306882 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-75qqk_96a24813-99cc-493c-84d1-5c9872de996a/payload-pre-processing/0.log" Jul 07 00:38:38.434120 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:38.434059 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-svzf4_54a92b1d-f008-412d-a789-da0f3bad7dd7/payload-processing/0.log" Jul 07 00:38:38.923275 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:38.923248 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-ftrr7_9edd6546-f7a3-47d3-b7c9-bc3cea61599b/main/0.log" Jul 07 00:38:39.408454 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:39.408405 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6v6ks_eeb6dd09-076a-4f33-9758-dfe4b7d35237/main/0.log" Jul 07 00:38:47.071127 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:47.071095 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9vrmj_7888a0e8-ee58-4d66-8720-70feb41fef79/global-pull-secret-syncer/0.log" Jul 07 00:38:47.350045 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:47.349955 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wgncd_e87a7305-1f60-4a49-bff3-2a0082e2c112/konnectivity-agent/0.log" Jul 07 00:38:47.408848 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:47.408815 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-52.ec2.internal_73905cf8fbe88f74cd23a2f960dd19a6/haproxy/0.log" Jul 07 00:38:51.809396 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:51.809360 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c_4c2be1b3-a140-4e94-8981-b9f8d257846f/extract/0.log" Jul 07 00:38:51.841492 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:51.841457 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c_4c2be1b3-a140-4e94-8981-b9f8d257846f/util/0.log" Jul 07 00:38:51.863623 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:51.863591 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592cj8c_4c2be1b3-a140-4e94-8981-b9f8d257846f/pull/0.log" Jul 07 00:38:51.894475 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:51.894446 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6_d63a0916-a7af-4d2b-941d-242ee4704ba0/extract/0.log" Jul 07 00:38:51.915625 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:51.915598 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6_d63a0916-a7af-4d2b-941d-242ee4704ba0/util/0.log" Jul 07 00:38:51.937265 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:51.937236 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0chlk6_d63a0916-a7af-4d2b-941d-242ee4704ba0/pull/0.log" Jul 07 00:38:51.962164 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:51.962133 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf_ed65c1a6-9419-47ac-ba01-8d7dc007b18f/extract/0.log" Jul 07 00:38:51.990671 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:51.990635 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf_ed65c1a6-9419-47ac-ba01-8d7dc007b18f/util/0.log" Jul 07 00:38:52.013193 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:52.013169 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73862lf_ed65c1a6-9419-47ac-ba01-8d7dc007b18f/pull/0.log" Jul 07 00:38:52.045721 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:52.045685 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz_4c1fb982-b6f5-4687-98c5-e6207076c890/extract/0.log" Jul 07 00:38:52.069392 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:52.069305 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz_4c1fb982-b6f5-4687-98c5-e6207076c890/util/0.log" Jul 07 00:38:52.092856 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:52.092826 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qmxhz_4c1fb982-b6f5-4687-98c5-e6207076c890/pull/0.log" Jul 07 00:38:54.637227 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:54.637193 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-dvlk6_93151fb7-0fa3-4ce4-ba37-78cd690e6c6b/kube-state-metrics/0.log" Jul 07 00:38:54.655399 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:54.655371 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-dvlk6_93151fb7-0fa3-4ce4-ba37-78cd690e6c6b/kube-rbac-proxy-main/0.log" Jul 07 00:38:54.676959 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:54.676872 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-dvlk6_93151fb7-0fa3-4ce4-ba37-78cd690e6c6b/kube-rbac-proxy-self/0.log" Jul 07 00:38:54.925418 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:54.925391 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x2ghg_6f5fa57d-9dae-461c-a129-dda4f8f24e95/node-exporter/0.log" Jul 07 00:38:54.949817 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:54.949738 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x2ghg_6f5fa57d-9dae-461c-a129-dda4f8f24e95/kube-rbac-proxy/0.log" Jul 07 00:38:54.973009 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:54.972979 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x2ghg_6f5fa57d-9dae-461c-a129-dda4f8f24e95/init-textfile/0.log" Jul 07 00:38:55.341452 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:55.341365 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59bd85dd-64knq_21a28c8f-7194-4947-9842-26b99034fd8a/telemeter-client/0.log" Jul 07 00:38:55.363219 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:55.363193 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59bd85dd-64knq_21a28c8f-7194-4947-9842-26b99034fd8a/reload/0.log" Jul 07 00:38:55.386201 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:55.386171 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59bd85dd-64knq_21a28c8f-7194-4947-9842-26b99034fd8a/kube-rbac-proxy/0.log" Jul 07 00:38:57.251520 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.251483 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp"] Jul 07 00:38:57.255091 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.252751 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a694d420-c2dc-4f2e-8ee6-c9f96509adb6" containerName="maas-api" Jul 07 00:38:57.255091 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.252786 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a694d420-c2dc-4f2e-8ee6-c9f96509adb6" containerName="maas-api" Jul 07 00:38:57.255091 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.252808 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="288cd1bf-89a4-4adb-8d82-26b1043a50f5" containerName="maas-api" Jul 07 00:38:57.255091 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.252827 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="288cd1bf-89a4-4adb-8d82-26b1043a50f5" containerName="maas-api" Jul 07 00:38:57.255091 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.253019 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="288cd1bf-89a4-4adb-8d82-26b1043a50f5" containerName="maas-api" Jul 07 00:38:57.255091 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.253040 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a694d420-c2dc-4f2e-8ee6-c9f96509adb6" containerName="maas-api" Jul 07 00:38:57.258343 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.258311 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.261319 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.261296 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7rzff\"/\"kube-root-ca.crt\"" Jul 07 00:38:57.262282 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.262254 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7rzff\"/\"default-dockercfg-h4m67\"" Jul 07 00:38:57.262408 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.262261 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7rzff\"/\"openshift-service-ca.crt\"" Jul 07 00:38:57.265537 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.265511 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp"] Jul 07 00:38:57.312888 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.312848 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-podres\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.313095 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.312912 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-sys\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.313095 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.312954 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-lib-modules\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.313095 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.312987 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78fkc\" (UniqueName: \"kubernetes.io/projected/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-kube-api-access-78fkc\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.313095 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.313022 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-proc\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.414322 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.414282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-podres\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.414504 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.414343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-sys\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.414504 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.414372 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-lib-modules\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.414504 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.414409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78fkc\" (UniqueName: \"kubernetes.io/projected/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-kube-api-access-78fkc\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.414504 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.414430 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-proc\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.414504 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.414485 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-podres\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.414669 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.414501 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-sys\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.414669 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.414526 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-proc\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.414669 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.414550 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-lib-modules\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.422895 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.422860 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78fkc\" (UniqueName: \"kubernetes.io/projected/2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6-kube-api-access-78fkc\") pod \"perf-node-gather-daemonset-9f7wp\" (UID: \"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.570886 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.570786 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.708157 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.708124 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp"] Jul 07 00:38:57.710077 ip-10-0-130-52 kubenswrapper[2562]: W0707 00:38:57.710038 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2ed7d284_e9f2_4fd4_b58d_d4d74a4539e6.slice/crio-165b97d6b7a3206bbfe8fa825097cfc1b30102e34c384386876e9f13c571d96c WatchSource:0}: Error finding container 165b97d6b7a3206bbfe8fa825097cfc1b30102e34c384386876e9f13c571d96c: Status 404 returned error can't find the container with id 165b97d6b7a3206bbfe8fa825097cfc1b30102e34c384386876e9f13c571d96c Jul 07 00:38:57.844300 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.844198 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" event={"ID":"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6","Type":"ContainerStarted","Data":"5e874ac1bd74c3c954500c1e56c4ca631d50e070efc202461d5a1de9b512c58e"} Jul 07 00:38:57.844300 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.844256 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:38:57.844300 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.844272 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" event={"ID":"2ed7d284-e9f2-4fd4-b58d-d4d74a4539e6","Type":"ContainerStarted","Data":"165b97d6b7a3206bbfe8fa825097cfc1b30102e34c384386876e9f13c571d96c"} Jul 07 00:38:57.862441 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:57.862390 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" podStartSLOduration=0.86237347 podStartE2EDuration="862.37347ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 00:38:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:38:57.860220182 +0000 UTC m=+2503.832704856" watchObservedRunningTime="2026-07-07 00:38:57.86237347 +0000 UTC m=+2503.834858130" Jul 07 00:38:59.593129 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:59.593053 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xhjt_ed424306-bd32-44c0-ac6b-354738a1eedd/dns/0.log" Jul 07 00:38:59.613443 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:59.613411 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xhjt_ed424306-bd32-44c0-ac6b-354738a1eedd/kube-rbac-proxy/0.log" Jul 07 00:38:59.759691 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:38:59.759659 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d5v6t_43039088-05c7-4888-ac30-35d9d2faee8b/dns-node-resolver/0.log" Jul 07 00:39:00.284377 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:00.284344 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29723040-6dzvd_57e0b7f3-6007-402f-8624-d7edad486846/image-pruner/0.log" Jul 07 00:39:00.332213 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:00.332184 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c8f8d46d6-hmqcr_f42f2861-0d59-4ac9-a6c7-a955b86059af/registry/0.log" Jul 07 00:39:00.381149 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:00.381113 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bfwh4_16c70b55-5ac2-4cd0-8f14-22846a7dbe7a/node-ca/0.log" Jul 07 00:39:01.542609 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:01.542575 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-59f7c_2b56aae5-875d-4c1f-82e7-70e360f432b3/discovery/0.log" Jul 07 00:39:01.700383 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:01.700341 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-75qqk_96a24813-99cc-493c-84d1-5c9872de996a/payload-pre-processing/0.log" Jul 07 00:39:01.778692 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:01.778653 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-svzf4_54a92b1d-f008-412d-a789-da0f3bad7dd7/payload-processing/0.log" Jul 07 00:39:02.528377 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:02.528330 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dprmh_9d914f67-2aac-4cca-91ea-77dd7f0eca61/serve-healthcheck-canary/0.log" Jul 07 00:39:03.085144 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:03.085114 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tr5kc_05598b83-7ee6-42f2-977c-a41b3738a4b7/kube-rbac-proxy/0.log" Jul 07 00:39:03.108759 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:03.108729 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tr5kc_05598b83-7ee6-42f2-977c-a41b3738a4b7/exporter/0.log" Jul 07 00:39:03.130803 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:03.130773 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tr5kc_05598b83-7ee6-42f2-977c-a41b3738a4b7/extractor/0.log" Jul 07 00:39:03.860356 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:03.860325 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-9f7wp" Jul 07 00:39:05.598684 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:05.598641 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5868db4bd6-qvcsh_8623a11e-31f3-40aa-b164-3781d41596e0/manager/0.log" Jul 07 00:39:05.742572 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:05.742471 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b6d6b477c-4trmc_09ebb26f-ac7b-4a20-be81-608ac47a4038/manager/0.log" Jul 07 00:39:05.768188 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:05.768162 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b6d6b477c-4trmc_09ebb26f-ac7b-4a20-be81-608ac47a4038/copy-manifests/0.log" Jul 07 00:39:07.344058 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:07.344026 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5df8c9476f-mnjh4_0ae3b8ac-6c91-4490-bc07-ff1a7622f8a3/manager/0.log" Jul 07 00:39:08.061666 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:08.061625 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-59f7c_2b56aae5-875d-4c1f-82e7-70e360f432b3/discovery/0.log" Jul 07 00:39:08.223875 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:08.223815 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-75qqk_96a24813-99cc-493c-84d1-5c9872de996a/payload-pre-processing/0.log" Jul 07 00:39:08.300485 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:08.300453 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-svzf4_54a92b1d-f008-412d-a789-da0f3bad7dd7/payload-processing/0.log" Jul 07 00:39:09.705032 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:09.704997 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-59f7c_2b56aae5-875d-4c1f-82e7-70e360f432b3/discovery/0.log" Jul 07 00:39:09.828590 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:09.828561 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-75qqk_96a24813-99cc-493c-84d1-5c9872de996a/payload-pre-processing/0.log" Jul 07 00:39:09.897227 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:09.897194 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-svzf4_54a92b1d-f008-412d-a789-da0f3bad7dd7/payload-processing/0.log" Jul 07 00:39:12.519530 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:12.519497 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-bn52k_fe3dbf5d-804e-4447-8bec-408cfe773dbf/migrator/0.log" Jul 07 00:39:12.540148 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:12.540116 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-bn52k_fe3dbf5d-804e-4447-8bec-408cfe773dbf/graceful-termination/0.log" Jul 07 00:39:14.163810 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:14.163782 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bvddn_354c35b5-4b7a-47c3-825e-ce0decc20f9a/kube-multus-additional-cni-plugins/0.log" Jul 07 00:39:14.185734 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:14.185700 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bvddn_354c35b5-4b7a-47c3-825e-ce0decc20f9a/egress-router-binary-copy/0.log" Jul 07 00:39:14.207488 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:14.207456 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bvddn_354c35b5-4b7a-47c3-825e-ce0decc20f9a/cni-plugins/0.log" Jul 07 00:39:14.228349 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:14.228323 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bvddn_354c35b5-4b7a-47c3-825e-ce0decc20f9a/bond-cni-plugin/0.log" Jul 07 00:39:14.248937 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:14.248910 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bvddn_354c35b5-4b7a-47c3-825e-ce0decc20f9a/routeoverride-cni/0.log" Jul 07 00:39:14.274245 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:14.274216 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bvddn_354c35b5-4b7a-47c3-825e-ce0decc20f9a/whereabouts-cni-bincopy/0.log" Jul 07 00:39:14.295893 ip-10-0-130-52 kubenswrapper[2562]: I0707 00:39:14.295865 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bvddn_354c35b5-4b7a-47c3-825e-ce0decc20f9a/whereabouts-cni/0.log"