Jul 02 19:00:35.278018 ip-10-0-138-101 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 02 19:00:35.278028 ip-10-0-138-101 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 02 19:00:35.278036 ip-10-0-138-101 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 02 19:00:35.278272 ip-10-0-138-101 systemd[1]: Failed to start Kubernetes Kubelet. Jul 02 19:00:46.736924 ip-10-0-138-101 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 02 19:00:46.736941 ip-10-0-138-101 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 21e13d53888a49f5a8b216ef57babcf4 -- Jul 02 19:02:42.596389 ip-10-0-138-101 systemd[1]: Starting Kubernetes Kubelet... Jul 02 19:02:42.992579 ip-10-0-138-101 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:02:42.992579 ip-10-0-138-101 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 19:02:42.992579 ip-10-0-138-101 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:02:42.992579 ip-10-0-138-101 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 19:02:42.992579 ip-10-0-138-101 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:02:42.994286 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:42.994164 2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 19:02:43.001917 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001897 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:02:43.001917 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001918 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001922 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001926 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001929 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001933 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001936 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001939 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001942 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001945 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001948 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001951 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001955 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001960 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001963 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001966 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001969 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001979 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001982 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001985 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001987 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:02:43.001990 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001990 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001993 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001996 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.001999 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002002 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002005 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002009 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002011 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002014 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002017 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002020 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002022 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002025 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002028 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002030 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002033 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002036 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002039 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002042 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002045 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:02:43.002518 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002048 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002050 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002060 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002064 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002067 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002069 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002072 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002074 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002077 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002081 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002083 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002086 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002089 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002091 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002096 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002099 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002102 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002104 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002107 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:02:43.003034 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002112 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002116 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002119 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002122 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002126 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002129 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002132 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002135 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002138 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002140 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002143 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002164 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002167 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002170 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002173 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002178 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002181 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002183 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002186 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002189 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:02:43.003548 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002192 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002195 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002197 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002201 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002205 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002209 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002212 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002216 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002219 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002222 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002226 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002228 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002231 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002234 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002237 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002240 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002243 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002246 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.002249 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:02:43.004061 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003011 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003017 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003020 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003023 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003026 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003029 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003032 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003035 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003038 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003041 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003043 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003046 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003049 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003052 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003054 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003057 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003060 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003064 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003066 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003069 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:02:43.004557 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003073 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003076 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003078 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003082 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003084 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003087 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003090 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003093 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003096 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003098 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003101 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003103 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003106 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003109 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003111 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003114 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003117 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003120 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003122 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:02:43.005086 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003125 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003128 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003131 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003136 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003141 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003161 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003165 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003168 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003173 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003176 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003179 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003182 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003186 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003189 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003192 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003194 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003197 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003200 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003203 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003206 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:02:43.005590 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003208 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003212 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003214 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003217 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003220 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003223 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003226 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003229 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003231 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003234 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003237 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003240 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003243 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003245 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003248 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003253 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003257 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003261 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003264 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:02:43.006111 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003267 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003270 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003273 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003276 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003279 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003283 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003286 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003289 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003291 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003294 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003297 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003299 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003303 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003307 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003311 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003314 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003317 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003319 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003322 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003324 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:02:43.006636 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003327 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003883 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003897 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003902 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003905 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003908 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003911 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003914 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003919 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003923 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003926 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003929 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003932 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003935 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003938 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003941 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003944 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003948 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003951 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:02:43.007186 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003954 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003957 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003960 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003963 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003966 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003968 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003971 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003973 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003976 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003979 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003982 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003986 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003990 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003993 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.003997 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004001 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004004 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004007 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004009 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:02:43.007677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004012 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004015 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004018 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004022 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004025 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004028 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004030 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004033 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004036 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004039 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004041 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004044 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004047 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004050 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004053 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004055 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004058 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004061 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004063 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004067 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:02:43.008178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004070 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004072 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004075 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004078 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004080 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004083 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004086 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004089 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004092 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004095 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004099 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004101 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004104 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004107 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004110 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004112 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004115 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004118 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004120 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004140 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004142 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:02:43.008726 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004145 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004148 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004151 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004154 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004158 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004161 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004164 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004167 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004169 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004172 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004176 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004180 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004183 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004186 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004189 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004192 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004195 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004197 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004201 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004208 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:02:43.009283 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004211 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004296 2571 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004307 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004318 2571 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004325 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004330 2571 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004334 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004339 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004343 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004347 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004350 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004354 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004357 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004360 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004364 2571 flags.go:64] FLAG: --cgroup-root="" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004367 2571 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004370 2571 flags.go:64] FLAG: --client-ca-file="" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004374 2571 flags.go:64] FLAG: --cloud-provider="external" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004377 2571 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004381 2571 flags.go:64] FLAG: --cluster-domain="" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004384 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004388 2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004391 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004395 2571 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 19:02:43.009850 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004399 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004402 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004405 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004409 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004412 2571 flags.go:64] FLAG: --contention-profiling="false" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004415 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004419 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004425 2571 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004428 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004434 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004437 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004441 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004444 2571 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004447 2571 flags.go:64] FLAG: --enable-server="true" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004450 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004456 2571 flags.go:64] FLAG: --event-burst="100" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004459 2571 flags.go:64] FLAG: --event-qps="50" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004462 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004466 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004469 2571 flags.go:64] FLAG: --eviction-hard="" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004473 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004476 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004479 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004485 2571 flags.go:64] FLAG: --eviction-soft="" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004488 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 19:02:43.010472 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004491 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004495 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004498 2571 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004501 2571 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004504 2571 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004507 2571 flags.go:64] FLAG: --feature-gates="" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004511 2571 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004514 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004518 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004522 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004527 2571 flags.go:64] FLAG: --healthz-port="10248" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004531 2571 flags.go:64] FLAG: --help="false" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004536 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004540 2571 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004545 2571 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004548 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004552 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004557 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004560 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004563 2571 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004567 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004570 2571 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004573 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004577 2571 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 19:02:43.011135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004580 2571 flags.go:64] FLAG: --kube-reserved="" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004583 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004586 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004589 2571 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004593 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004596 2571 flags.go:64] FLAG: --lock-file="" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004601 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004604 2571 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004607 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004613 2571 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004616 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004619 2571 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004623 2571 flags.go:64] FLAG: --logging-format="text" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004626 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004629 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004632 2571 flags.go:64] FLAG: --manifest-url="" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004636 2571 flags.go:64] FLAG: --manifest-url-header="" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004641 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004644 2571 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004648 2571 flags.go:64] FLAG: --max-pods="110" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004651 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004655 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004659 2571 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004662 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004665 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 19:02:43.011822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004668 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004672 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004681 2571 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004684 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004687 2571 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004691 2571 flags.go:64] FLAG: --pod-cidr="" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004694 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004701 2571 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004704 2571 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004708 2571 flags.go:64] FLAG: --pods-per-core="0" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004711 2571 flags.go:64] FLAG: --port="10250" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004714 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004718 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0edd0266b780fad99" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004722 2571 flags.go:64] FLAG: --qos-reserved="" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004725 2571 flags.go:64] FLAG: --read-only-port="10255" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004729 2571 flags.go:64] FLAG: --register-node="true" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004732 2571 flags.go:64] FLAG: --register-with-taints="" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004742 2571 flags.go:64] FLAG: --registry-burst="10" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004745 2571 flags.go:64] FLAG: --registry-qps="5" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004748 2571 flags.go:64] FLAG: --reserved-cpus="" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004751 2571 flags.go:64] FLAG: --reserved-memory="" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004755 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004759 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004762 2571 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004765 2571 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 19:02:43.012513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004768 2571 flags.go:64] FLAG: --runonce="false" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004771 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004775 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004778 2571 flags.go:64] FLAG: --seccomp-default="false" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004782 2571 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004786 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004789 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004792 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004796 2571 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004799 2571 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004803 2571 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004806 2571 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004809 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004812 2571 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004815 2571 flags.go:64] FLAG: --system-cgroups="" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004818 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004824 2571 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004827 2571 flags.go:64] FLAG: --tls-cert-file="" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004830 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004835 2571 flags.go:64] FLAG: --tls-min-version="" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004839 2571 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004842 2571 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004845 2571 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004848 2571 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004851 2571 flags.go:64] FLAG: --v="2" Jul 02 19:02:43.013606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004856 2571 flags.go:64] FLAG: --version="false" Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004860 2571 flags.go:64] FLAG: --vmodule="" Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004864 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.004867 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004992 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004996 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.004999 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005003 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005006 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005009 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005012 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005015 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005019 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005022 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005025 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005028 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005030 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005034 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005037 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005042 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:02:43.014382 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005046 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005049 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005051 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005059 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005062 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005065 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005068 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005072 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005075 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005078 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005081 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005084 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005086 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005089 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005093 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005096 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005098 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005101 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005104 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:02:43.014912 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005107 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005110 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005113 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005116 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005118 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005122 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005125 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005128 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005131 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005134 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005137 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005140 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005143 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005160 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005163 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005166 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005169 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005172 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005174 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005177 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:02:43.015515 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005181 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005184 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005187 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005190 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005192 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005195 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005198 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005201 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005204 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005206 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005209 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005212 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005215 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005217 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005220 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005223 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005225 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005229 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005232 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005235 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:02:43.016032 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005238 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005240 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005243 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005246 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005249 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005252 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005255 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005258 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005261 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005263 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005268 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005272 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005276 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005280 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005283 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005285 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005288 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005291 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005294 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:02:43.016564 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005296 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005300 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005304 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005307 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.005310 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.006307 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.014824 2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.014845 2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.014866 2571 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.014872 2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014938 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014944 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014949 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014954 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:02:43.017064 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014957 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014961 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014964 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014968 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014972 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014975 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014979 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014983 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014987 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014990 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014993 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014996 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.014999 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015002 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015005 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015007 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015010 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015013 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015016 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:02:43.017460 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015018 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015022 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015026 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015029 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015032 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015035 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015038 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015042 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015045 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015048 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015051 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015063 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015066 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015069 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015072 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015075 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015078 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015080 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015083 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:02:43.017961 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015086 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015089 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015092 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015095 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015099 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015102 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015105 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015108 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015111 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015113 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015116 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015119 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015122 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015124 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015127 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015130 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015133 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015136 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015138 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015142 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:02:43.018479 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015146 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015163 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015166 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015169 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015171 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015174 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015177 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015180 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015182 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015185 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015188 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015191 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015194 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015196 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015199 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015202 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015205 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015208 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015211 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015214 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:02:43.019097 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015217 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015220 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015222 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015225 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015228 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015231 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015233 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015237 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015239 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015242 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015245 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015249 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015252 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015255 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015258 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015261 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:02:43.019644 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015264 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.015270 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015395 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015400 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015403 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015407 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015410 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015413 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015416 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015419 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015422 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015425 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015428 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015433 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:02:43.020058 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015438 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015441 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015444 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015447 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015450 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015454 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015457 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015460 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015463 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015466 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015469 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015472 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015475 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015478 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015482 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015485 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015488 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015491 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015493 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015496 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:02:43.020489 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015499 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015502 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015504 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015507 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015510 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015513 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015516 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015519 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015523 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015527 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015529 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015532 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015535 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015538 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015541 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015543 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015546 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015549 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015552 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015554 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:02:43.021008 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015557 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015560 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015562 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015565 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015568 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015571 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015574 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015577 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015579 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015582 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015585 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015587 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015590 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015593 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015596 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015598 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015601 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015604 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015606 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015609 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:02:43.021541 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015612 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015614 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015618 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015622 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015625 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015628 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015632 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015634 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015638 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015641 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015644 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015647 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015650 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015652 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015655 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015658 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015661 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015664 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015666 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:02:43.022059 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015669 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015672 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015674 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015677 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015680 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015682 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015685 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:43.015687 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.015693 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.015819 2571 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.018115 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.019782 2571 server.go:1019] "Starting client certificate rotation" Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.019898 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 19:02:43.022592 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.019941 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 19:02:43.022981 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.022293 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 19:02:43.023649 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.023626 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 19:02:43.034484 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.034466 2571 log.go:25] "Validated CRI v1 runtime API" Jul 02 19:02:43.036842 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.036829 2571 log.go:25] "Validated CRI v1 image API" Jul 02 19:02:43.040662 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.040631 2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 19:02:43.043522 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.043501 2571 fs.go:135] Filesystem UUIDs: map[2f50d770-6f1e-4e48-bd8f-fbb441ecf6c8:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9279738d-52b6-4088-bb22-b038730d62a2:/dev/nvme0n1p4] Jul 02 19:02:43.043611 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.043520 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 19:02:43.049971 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.049838 2571 manager.go:216] Machine: {Timestamp:2026-07-02 19:02:43.048375723 +0000 UTC m=+0.339712617 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107459 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bb4f79c124f3fd0c746e53ff5aaaf SystemUUID:ec2bb4f7-9c12-4f3f-d0c7-46e53ff5aaaf BootID:21e13d53-888a-49f5-a8b2-16ef57babcf4 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:66:3e:ae:a2:41 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:66:3e:ae:a2:41 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:b7:00:f9:67:9b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 19:02:43.049971 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.049965 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 19:02:43.050123 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.050103 2571 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 19:02:43.053883 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.053852 2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 19:02:43.054100 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.053886 2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-101.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 19:02:43.054232 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.054216 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 19:02:43.054735 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.054720 2571 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 19:02:43.054792 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.054742 2571 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 19:02:43.054792 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.054755 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 19:02:43.055339 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.055327 2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 19:02:43.057194 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.057181 2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 19:02:43.060258 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.060242 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 02 19:02:43.060403 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.060391 2571 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 19:02:43.060461 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.060449 2571 kubelet.go:486] "Attempting to sync node with API server" Jul 02 19:02:43.060524 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.060464 2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 19:02:43.060524 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.060483 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 19:02:43.060524 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.060497 2571 kubelet.go:398] "Adding apiserver pod source" Jul 02 19:02:43.060661 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.060567 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 19:02:43.061069 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.061055 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 19:02:43.061160 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.061095 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 19:02:43.065475 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.065457 2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 19:02:43.067427 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.067411 2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 19:02:43.067491 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.067461 2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 19:02:43.068611 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068597 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 19:02:43.068680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068617 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 19:02:43.068680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068624 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 19:02:43.068680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068630 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 19:02:43.068680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068639 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 19:02:43.068680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068645 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 19:02:43.068680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068654 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 19:02:43.068680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068659 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 19:02:43.068680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068666 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 19:02:43.068680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068672 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 19:02:43.068680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068681 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 19:02:43.068680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.068687 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 19:02:43.070207 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.070197 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 19:02:43.070207 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.070207 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 19:02:43.073641 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.073622 2571 server.go:1268] "Started kubelet" Jul 02 19:02:43.073717 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.073657 2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-101.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 19:02:43.073807 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.073737 2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 19:02:43.073877 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.073833 2571 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 19:02:43.074235 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.074211 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 19:02:43.074368 ip-10-0-138-101 systemd[1]: Started Kubernetes Kubelet. Jul 02 19:02:43.074474 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.074373 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 19:02:43.074474 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.074404 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 19:02:43.075572 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.075552 2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 19:02:43.076452 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.076436 2571 server.go:310] "Adding debug handlers to kubelet server" Jul 02 19:02:43.080882 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.080860 2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 19:02:43.082142 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.082125 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 19:02:43.082224 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.082137 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 19:02:43.082318 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.082300 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:43.082874 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.081888 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-101.ec2.internal.18be8e4cd1c9676d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-101.ec2.internal,UID:ip-10-0-138-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-101.ec2.internal,},FirstTimestamp:2026-07-02 19:02:43.073591149 +0000 UTC m=+0.364928047,LastTimestamp:2026-07-02 19:02:43.073591149 +0000 UTC m=+0.364928047,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-101.ec2.internal,}" Jul 02 19:02:43.082984 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.082900 2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 19:02:43.083206 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083182 2571 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 19:02:43.083206 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083206 2571 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 19:02:43.083368 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083267 2571 factory.go:153] Registering CRI-O factory Jul 02 19:02:43.083368 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083281 2571 factory.go:223] Registration of the crio container factory successfully Jul 02 19:02:43.083368 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083337 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 19:02:43.083368 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083345 2571 factory.go:55] Registering systemd factory Jul 02 19:02:43.083368 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083353 2571 factory.go:223] Registration of the systemd container factory successfully Jul 02 19:02:43.083616 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083378 2571 factory.go:103] Registering Raw factory Jul 02 19:02:43.083616 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083391 2571 manager.go:1196] Started watching for new ooms in manager Jul 02 19:02:43.083616 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083335 2571 reconstruct.go:97] "Volume reconstruction finished" Jul 02 19:02:43.083616 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083417 2571 reconciler.go:29] "Reconciler: start to sync state" Jul 02 19:02:43.083869 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.083853 2571 manager.go:349] Starting recovery of all containers Jul 02 19:02:43.086200 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.086175 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2grgx" Jul 02 19:02:43.089045 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.088423 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 19:02:43.089045 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.088555 2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 19:02:43.094799 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.094769 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2grgx" Jul 02 19:02:43.099641 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.099609 2571 manager.go:354] Recovery completed Jul 02 19:02:43.100977 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.100948 2571 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 02 19:02:43.104330 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.104274 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:02:43.106997 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.106980 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:02:43.107093 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.107013 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:02:43.107093 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.107027 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:02:43.107625 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.107606 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 19:02:43.107625 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.107623 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 19:02:43.107781 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.107644 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 02 19:02:43.107781 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.107615 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:02:43.108374 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.108359 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:02:43.108434 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.108386 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:02:43.108434 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.108396 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:02:43.108923 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.108909 2571 policy_none.go:49] "None policy: Start" Jul 02 19:02:43.108986 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.108929 2571 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 19:02:43.108986 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.108944 2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 19:02:43.109489 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.109397 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-101.ec2.internal.18be8e4cd3c721de default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-101.ec2.internal,UID:ip-10-0-138-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-101.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-101.ec2.internal,},FirstTimestamp:2026-07-02 19:02:43.106996702 +0000 UTC m=+0.398333599,LastTimestamp:2026-07-02 19:02:43.106996702 +0000 UTC m=+0.398333599,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-101.ec2.internal,}" Jul 02 19:02:43.110184 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.110171 2571 policy_none.go:47] "Start" Jul 02 19:02:43.150862 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.150798 2571 manager.go:337] "Starting Device Plugin manager" Jul 02 19:02:43.150862 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.150843 2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 19:02:43.150862 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.150857 2571 server.go:87] "Starting device plugin registration server" Jul 02 19:02:43.151344 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.151193 2571 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 19:02:43.151435 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.151351 2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 19:02:43.151486 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.151465 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 19:02:43.151633 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.151599 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 19:02:43.151633 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.151617 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 19:02:43.152271 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.152250 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 19:02:43.152346 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.152311 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:43.229483 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.229440 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 19:02:43.230835 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.230818 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 19:02:43.230913 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.230860 2571 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 19:02:43.230913 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.230889 2571 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 19:02:43.231021 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.230936 2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 19:02:43.233622 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.233598 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:02:43.252029 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.252011 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:02:43.252982 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.252967 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:02:43.253051 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.253002 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:02:43.253051 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.253013 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:02:43.253051 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.253037 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.262006 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.261989 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.262065 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.262011 2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-101.ec2.internal\": node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:43.277083 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.277056 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:43.332425 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.332388 2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal"] Jul 02 19:02:43.332532 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.332483 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:02:43.333511 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.333491 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:02:43.333607 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.333523 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:02:43.333607 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.333534 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:02:43.334769 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.334756 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:02:43.334884 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.334868 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.334921 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.334903 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:02:43.335537 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.335522 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:02:43.335607 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.335546 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:02:43.335607 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.335558 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:02:43.336174 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.336157 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:02:43.336256 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.336191 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:02:43.336256 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.336206 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:02:43.337389 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.337371 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.337484 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.337401 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:02:43.338122 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.338104 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:02:43.338214 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.338137 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:02:43.338214 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.338151 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:02:43.359896 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.359869 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-101.ec2.internal\" not found" node="ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.364542 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.364525 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-101.ec2.internal\" not found" node="ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.377131 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.377105 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:43.387543 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.387520 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9ccd326e5056db2ec15b8f22d5755cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"a9ccd326e5056db2ec15b8f22d5755cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.387647 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.387566 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ccd326e5056db2ec15b8f22d5755cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"a9ccd326e5056db2ec15b8f22d5755cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.387647 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.387587 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9b6c8b6d42e5b52ab4cf9f706a9be0f7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-101.ec2.internal\" (UID: \"9b6c8b6d42e5b52ab4cf9f706a9be0f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.478089 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.478003 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:43.488169 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.488147 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9ccd326e5056db2ec15b8f22d5755cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"a9ccd326e5056db2ec15b8f22d5755cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.488223 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.488176 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ccd326e5056db2ec15b8f22d5755cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"a9ccd326e5056db2ec15b8f22d5755cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.488223 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.488203 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9b6c8b6d42e5b52ab4cf9f706a9be0f7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-101.ec2.internal\" (UID: \"9b6c8b6d42e5b52ab4cf9f706a9be0f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.488325 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.488250 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9b6c8b6d42e5b52ab4cf9f706a9be0f7-config\") pod \"kube-apiserver-proxy-ip-10-0-138-101.ec2.internal\" (UID: \"9b6c8b6d42e5b52ab4cf9f706a9be0f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.488325 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.488254 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9ccd326e5056db2ec15b8f22d5755cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"a9ccd326e5056db2ec15b8f22d5755cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.488325 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.488275 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9ccd326e5056db2ec15b8f22d5755cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal\" (UID: \"a9ccd326e5056db2ec15b8f22d5755cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.578963 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.578921 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:43.668973 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.668928 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.669115 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:43.668979 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" Jul 02 19:02:43.679596 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.679569 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:43.780925 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.780826 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:43.881194 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.881153 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:43.981936 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:43.981901 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:44.020171 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.020141 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 19:02:44.020633 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.020309 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 19:02:44.082803 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:44.082725 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-101.ec2.internal\" not found" Jul 02 19:02:44.097556 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.097508 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 18:57:43 +0000 UTC" deadline="2027-12-05 06:53:07.342983821 +0000 UTC" Jul 02 19:02:44.097556 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.097542 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12491h50m23.245444665s" Jul 02 19:02:44.136769 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.136742 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:02:44.184640 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.184586 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" Jul 02 19:02:44.197839 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.197814 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 19:02:44.199731 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.199716 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" Jul 02 19:02:44.208571 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.208552 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 19:02:44.291764 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.291729 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:02:44.534640 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:44.534595 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6c8b6d42e5b52ab4cf9f706a9be0f7.slice/crio-165950781c015f118ca299a4a46fa41ca0d595db7919efb703114d4c199f681e WatchSource:0}: Error finding container 165950781c015f118ca299a4a46fa41ca0d595db7919efb703114d4c199f681e: Status 404 returned error can't find the container with id 165950781c015f118ca299a4a46fa41ca0d595db7919efb703114d4c199f681e Jul 02 19:02:44.534952 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:44.534933 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ccd326e5056db2ec15b8f22d5755cf.slice/crio-90f70bfb6db9d86c546cfdbe0e3de8c1c1628e336121ac0b18db80c514d7d1c2 WatchSource:0}: Error finding container 90f70bfb6db9d86c546cfdbe0e3de8c1c1628e336121ac0b18db80c514d7d1c2: Status 404 returned error can't find the container with id 90f70bfb6db9d86c546cfdbe0e3de8c1c1628e336121ac0b18db80c514d7d1c2 Jul 02 19:02:44.539969 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.539951 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:02:44.688721 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:44.688617 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:02:45.064296 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.064207 2571 apiserver.go:52] "Watching apiserver" Jul 02 19:02:45.071408 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.071381 2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 19:02:45.072480 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.072458 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-z6zb2","kube-system/global-pull-secret-syncer-8sprk","kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal","openshift-multus/multus-7tb99","openshift-multus/multus-additional-cni-plugins-9z9qb","openshift-network-operator/iptables-alerter-c9rtw","openshift-ovn-kubernetes/ovnkube-node-kr7sw","kube-system/konnectivity-agent-kkggr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc","openshift-cluster-node-tuning-operator/tuned-ckk29","openshift-dns/node-resolver-v2xxk","openshift-image-registry/node-ca-rv7zv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal","openshift-multus/network-metrics-daemon-27pfj"] Jul 02 19:02:45.075471 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.075454 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.076883 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.076858 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v2xxk" Jul 02 19:02:45.078168 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.078143 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 19:02:45.078267 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.078164 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 19:02:45.078342 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.078324 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 19:02:45.078608 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.078591 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:45.078727 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.078702 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:02:45.078895 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.078869 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 19:02:45.079007 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.078874 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 19:02:45.079304 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.079286 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-88g9k\"" Jul 02 19:02:45.079489 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.079474 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 19:02:45.079566 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.079496 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 19:02:45.080161 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.080141 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 19:02:45.080265 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.080146 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-srlf6\"" Jul 02 19:02:45.080331 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.080270 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.082238 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.082222 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.082555 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.082535 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 19:02:45.082555 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.082539 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 19:02:45.082686 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.082561 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-m5v75\"" Jul 02 19:02:45.082686 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.082632 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 19:02:45.082686 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.082674 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 19:02:45.082839 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.082787 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 19:02:45.084811 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.084786 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c9rtw" Jul 02 19:02:45.084913 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.084824 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 19:02:45.085551 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.085514 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jrfbx\"" Jul 02 19:02:45.085551 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.085528 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 19:02:45.087356 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.087324 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:02:45.088015 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.087993 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 19:02:45.088140 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.088030 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zbqwz\"" Jul 02 19:02:45.088140 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.088135 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 19:02:45.088254 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.088225 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.090196 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.090176 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8xlqd\"" Jul 02 19:02:45.090335 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.090318 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:02:45.090487 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.090471 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 19:02:45.090742 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.090726 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 19:02:45.090874 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.090861 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 19:02:45.092418 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.092396 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:45.092519 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.092485 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:02:45.092702 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.092624 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 19:02:45.092776 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.092756 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7kzlv\"" Jul 02 19:02:45.092835 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.092778 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 19:02:45.094697 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.094677 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.095122 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.095104 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 19:02:45.096511 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096495 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-etc-openvswitch\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.096594 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096518 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-run-openvswitch\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.096594 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096535 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-log-socket\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.096594 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096550 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-cni-netd\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.096594 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096566 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9nz8\" (UniqueName: \"kubernetes.io/projected/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-kube-api-access-n9nz8\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.096814 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096624 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-socket-dir-parent\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.096814 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096679 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-systemd-units\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.096814 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096710 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.096814 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096732 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89964\" (UniqueName: \"kubernetes.io/projected/fe8227ad-efbc-48d6-bf5b-21902eeeb40f-kube-api-access-89964\") pod \"node-resolver-v2xxk\" (UID: \"fe8227ad-efbc-48d6-bf5b-21902eeeb40f\") " pod="openshift-dns/node-resolver-v2xxk" Jul 02 19:02:45.096814 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096787 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-cnibin\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096816 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096823 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-system-cni-dir\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096844 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jhwbd\"" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096866 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb60b6fe-625d-4029-9082-cdd088ab2179-cni-binary-copy\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096906 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28hdr\" (UniqueName: \"kubernetes.io/projected/cb60b6fe-625d-4029-9082-cdd088ab2179-kube-api-access-28hdr\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096924 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096944 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a2cd2a0-f19d-4516-8937-144762938c44-iptables-alerter-script\") pod \"iptables-alerter-c9rtw\" (UID: \"9a2cd2a0-f19d-4516-8937-144762938c44\") " pod="openshift-network-operator/iptables-alerter-c9rtw" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096968 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a2cd2a0-f19d-4516-8937-144762938c44-host-slash\") pod \"iptables-alerter-c9rtw\" (UID: \"9a2cd2a0-f19d-4516-8937-144762938c44\") " pod="openshift-network-operator/iptables-alerter-c9rtw" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.096993 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk9qj\" (UniqueName: \"kubernetes.io/projected/9a2cd2a0-f19d-4516-8937-144762938c44-kube-api-access-mk9qj\") pod \"iptables-alerter-c9rtw\" (UID: \"9a2cd2a0-f19d-4516-8937-144762938c44\") " pod="openshift-network-operator/iptables-alerter-c9rtw" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097043 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-slash\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097073 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-run-netns\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.097089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097099 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-run-ovn\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097110 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097139 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fe8227ad-efbc-48d6-bf5b-21902eeeb40f-hosts-file\") pod \"node-resolver-v2xxk\" (UID: \"fe8227ad-efbc-48d6-bf5b-21902eeeb40f\") " pod="openshift-dns/node-resolver-v2xxk" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097162 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-cni-dir\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097198 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-kubelet\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097232 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-ovnkube-script-lib\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.097241 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097250 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-run-netns\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097286 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-var-lib-kubelet\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097312 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-hostroot\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097336 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-run-multus-certs\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097363 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvbtg\" (UniqueName: \"kubernetes.io/projected/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-kube-api-access-gvbtg\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097391 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-os-release\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097418 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe8227ad-efbc-48d6-bf5b-21902eeeb40f-tmp-dir\") pod \"node-resolver-v2xxk\" (UID: \"fe8227ad-efbc-48d6-bf5b-21902eeeb40f\") " pod="openshift-dns/node-resolver-v2xxk" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097455 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-run-systemd\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097480 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-var-lib-openvswitch\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097504 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-run-ovn-kubernetes\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.097620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097536 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-cni-bin\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097565 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-ovnkube-config\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097587 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-ovn-node-metrics-cert\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097623 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-system-cni-dir\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097659 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgswk\" (UniqueName: \"kubernetes.io/projected/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-kube-api-access-tgswk\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097682 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-cni-binary-copy\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097716 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-run-k8s-cni-cncf-io\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097742 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-var-lib-cni-bin\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097771 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-var-lib-cni-multus\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097796 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-env-overrides\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097819 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097845 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-os-release\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097870 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-daemon-config\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097894 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-etc-kubernetes\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097913 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-cnibin\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097929 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097945 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-node-log\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.098136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097960 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-conf-dir\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.098708 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.097974 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb60b6fe-625d-4029-9082-cdd088ab2179-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.098708 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.098006 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb60b6fe-625d-4029-9082-cdd088ab2179-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.099097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.099084 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rv7zv" Jul 02 19:02:45.101029 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.101007 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 19:02:45.101379 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.101358 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 19:02:45.101486 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.101379 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 19:02:45.101486 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.101418 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f8kgh\"" Jul 02 19:02:45.116883 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.116854 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ntkfr" Jul 02 19:02:45.125114 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.125095 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ntkfr" Jul 02 19:02:45.183958 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.183927 2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 19:02:45.198205 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198175 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-slash\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.198387 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198216 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-run-netns\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.198387 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198245 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-run-ovn\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.198387 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198268 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-hostroot\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.198387 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198306 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-slash\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.198387 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198327 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-hostroot\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.198387 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198307 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-run-netns\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.198387 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198331 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-run-ovn\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.198387 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198331 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvbtg\" (UniqueName: \"kubernetes.io/projected/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-kube-api-access-gvbtg\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.198387 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198376 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz8ck\" (UniqueName: \"kubernetes.io/projected/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-kube-api-access-qz8ck\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198405 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198422 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-kubelet\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198458 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-run-netns\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198480 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-kubelet\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198494 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-var-lib-kubelet\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198520 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-run-multus-certs\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198553 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-var-lib-kubelet\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198600 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-modprobe-d\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198629 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198646 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe8227ad-efbc-48d6-bf5b-21902eeeb40f-tmp-dir\") pod \"node-resolver-v2xxk\" (UID: \"fe8227ad-efbc-48d6-bf5b-21902eeeb40f\") " pod="openshift-dns/node-resolver-v2xxk" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198662 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-run-multus-certs\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198665 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-run-netns\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198690 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-run-systemd\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198739 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-ovnkube-config\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198773 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-run-systemd\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198811 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-system-cni-dir\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.198843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198845 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-system-cni-dir\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198870 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-run-k8s-cni-cncf-io\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198893 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-var-lib-kubelet\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198918 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-etc-selinux\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198936 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-run-k8s-cni-cncf-io\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198955 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-cni-binary-copy\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198954 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe8227ad-efbc-48d6-bf5b-21902eeeb40f-tmp-dir\") pod \"node-resolver-v2xxk\" (UID: \"fe8227ad-efbc-48d6-bf5b-21902eeeb40f\") " pod="openshift-dns/node-resolver-v2xxk" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.198985 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-var-lib-cni-bin\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199009 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-var-lib-cni-multus\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199023 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-var-lib-cni-bin\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199034 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9332d088-8ecc-4c56-a0d8-6d35d5acda58-kubelet-config\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199051 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-sys\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199081 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eee2ced5-efa4-498e-83f9-793366d871b7-agent-certs\") pod \"konnectivity-agent-kkggr\" (UID: \"eee2ced5-efa4-498e-83f9-793366d871b7\") " pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199076 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-host-var-lib-cni-multus\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199122 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-node-log\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199150 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-node-log\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199156 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb60b6fe-625d-4029-9082-cdd088ab2179-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.199595 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199185 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb60b6fe-625d-4029-9082-cdd088ab2179-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199214 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-registration-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199245 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-etc-openvswitch\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199281 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-cni-netd\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199308 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-etc-openvswitch\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199311 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-socket-dir-parent\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199363 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-cni-netd\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199387 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-system-cni-dir\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199407 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-system-cni-dir\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199416 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-sysctl-conf\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199440 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-socket-dir-parent\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199463 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199492 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-cnibin\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199526 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-lib-modules\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199550 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-host\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199567 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-cnibin\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199574 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a2cd2a0-f19d-4516-8937-144762938c44-host-slash\") pod \"iptables-alerter-c9rtw\" (UID: \"9a2cd2a0-f19d-4516-8937-144762938c44\") " pod="openshift-network-operator/iptables-alerter-c9rtw" Jul 02 19:02:45.200280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199582 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-cni-binary-copy\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199599 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fe8227ad-efbc-48d6-bf5b-21902eeeb40f-hosts-file\") pod \"node-resolver-v2xxk\" (UID: \"fe8227ad-efbc-48d6-bf5b-21902eeeb40f\") " pod="openshift-dns/node-resolver-v2xxk" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199626 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-cni-dir\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199642 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a2cd2a0-f19d-4516-8937-144762938c44-host-slash\") pod \"iptables-alerter-c9rtw\" (UID: \"9a2cd2a0-f19d-4516-8937-144762938c44\") " pod="openshift-network-operator/iptables-alerter-c9rtw" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199582 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-ovnkube-config\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199652 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-sysconfig\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199694 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-kubernetes\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199741 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-socket-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199756 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb60b6fe-625d-4029-9082-cdd088ab2179-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199762 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb60b6fe-625d-4029-9082-cdd088ab2179-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199729 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fe8227ad-efbc-48d6-bf5b-21902eeeb40f-hosts-file\") pod \"node-resolver-v2xxk\" (UID: \"fe8227ad-efbc-48d6-bf5b-21902eeeb40f\") " pod="openshift-dns/node-resolver-v2xxk" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199762 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-device-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199807 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-cni-dir\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199815 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-ovnkube-script-lib\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199844 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-os-release\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199863 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbrlj\" (UniqueName: \"kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj\") pod \"network-check-target-z6zb2\" (UID: \"7c305fe4-c346-4c57-b6c6-1dea6f663801\") " pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199881 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eee2ced5-efa4-498e-83f9-793366d871b7-konnectivity-ca\") pod \"konnectivity-agent-kkggr\" (UID: \"eee2ced5-efa4-498e-83f9-793366d871b7\") " pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:02:45.200954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199899 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced-serviceca\") pod \"node-ca-rv7zv\" (UID: \"3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced\") " pod="openshift-image-registry/node-ca-rv7zv" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199922 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-var-lib-openvswitch\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199940 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-run-ovn-kubernetes\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199940 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-os-release\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199954 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-var-lib-openvswitch\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199955 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-cni-bin\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199988 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-cni-bin\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199993 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-run-ovn-kubernetes\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.199997 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-ovn-node-metrics-cert\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200037 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgswk\" (UniqueName: \"kubernetes.io/projected/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-kube-api-access-tgswk\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200066 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-sysctl-d\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200110 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-tuned\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200136 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-496gr\" (UniqueName: \"kubernetes.io/projected/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-kube-api-access-496gr\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200165 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-env-overrides\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200193 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.200298 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200300 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-os-release\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.201548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200331 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-daemon-config\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200349 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-ovnkube-script-lib\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.200364 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs podName:3afa8f46-c8f3-4b0b-a33e-42d05fefc58f nodeName:}" failed. No retries permitted until 2026-07-02 19:02:45.700343273 +0000 UTC m=+2.991680171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs") pod "network-metrics-daemon-27pfj" (UID: "3afa8f46-c8f3-4b0b-a33e-42d05fefc58f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200271 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200391 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-etc-kubernetes\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200416 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-cnibin\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200417 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-os-release\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200457 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200466 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb60b6fe-625d-4029-9082-cdd088ab2179-cnibin\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200487 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-sys-fs\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200495 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-etc-kubernetes\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200514 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-conf-dir\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200540 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-tmp\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200566 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtwv5\" (UniqueName: \"kubernetes.io/projected/3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced-kube-api-access-qtwv5\") pod \"node-ca-rv7zv\" (UID: \"3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced\") " pod="openshift-image-registry/node-ca-rv7zv" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200572 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-conf-dir\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200592 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-env-overrides\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200593 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-run-openvswitch\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.202097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200633 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-run-openvswitch\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200638 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-log-socket\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200665 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9nz8\" (UniqueName: \"kubernetes.io/projected/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-kube-api-access-n9nz8\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200680 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-log-socket\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200682 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-run\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200718 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced-host\") pod \"node-ca-rv7zv\" (UID: \"3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced\") " pod="openshift-image-registry/node-ca-rv7zv" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200765 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-multus-daemon-config\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200773 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-systemd-units\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200806 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200819 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-systemd-units\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200835 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89964\" (UniqueName: \"kubernetes.io/projected/fe8227ad-efbc-48d6-bf5b-21902eeeb40f-kube-api-access-89964\") pod \"node-resolver-v2xxk\" (UID: \"fe8227ad-efbc-48d6-bf5b-21902eeeb40f\") " pod="openshift-dns/node-resolver-v2xxk" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200852 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200859 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb60b6fe-625d-4029-9082-cdd088ab2179-cni-binary-copy\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200883 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28hdr\" (UniqueName: \"kubernetes.io/projected/cb60b6fe-625d-4029-9082-cdd088ab2179-kube-api-access-28hdr\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200905 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9332d088-8ecc-4c56-a0d8-6d35d5acda58-dbus\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200926 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-systemd\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200948 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a2cd2a0-f19d-4516-8937-144762938c44-iptables-alerter-script\") pod \"iptables-alerter-c9rtw\" (UID: \"9a2cd2a0-f19d-4516-8937-144762938c44\") " pod="openshift-network-operator/iptables-alerter-c9rtw" Jul 02 19:02:45.202813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.200974 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk9qj\" (UniqueName: \"kubernetes.io/projected/9a2cd2a0-f19d-4516-8937-144762938c44-kube-api-access-mk9qj\") pod \"iptables-alerter-c9rtw\" (UID: \"9a2cd2a0-f19d-4516-8937-144762938c44\") " pod="openshift-network-operator/iptables-alerter-c9rtw" Jul 02 19:02:45.203350 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.201309 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb60b6fe-625d-4029-9082-cdd088ab2179-cni-binary-copy\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.203350 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.201421 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a2cd2a0-f19d-4516-8937-144762938c44-iptables-alerter-script\") pod \"iptables-alerter-c9rtw\" (UID: \"9a2cd2a0-f19d-4516-8937-144762938c44\") " pod="openshift-network-operator/iptables-alerter-c9rtw" Jul 02 19:02:45.204091 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.204069 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-ovn-node-metrics-cert\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.207499 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.207481 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvbtg\" (UniqueName: \"kubernetes.io/projected/45980b15-071c-4aa7-a6e6-8ae7c9d48de2-kube-api-access-gvbtg\") pod \"multus-7tb99\" (UID: \"45980b15-071c-4aa7-a6e6-8ae7c9d48de2\") " pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.212987 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.212962 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89964\" (UniqueName: \"kubernetes.io/projected/fe8227ad-efbc-48d6-bf5b-21902eeeb40f-kube-api-access-89964\") pod \"node-resolver-v2xxk\" (UID: \"fe8227ad-efbc-48d6-bf5b-21902eeeb40f\") " pod="openshift-dns/node-resolver-v2xxk" Jul 02 19:02:45.213099 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.213022 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk9qj\" (UniqueName: \"kubernetes.io/projected/9a2cd2a0-f19d-4516-8937-144762938c44-kube-api-access-mk9qj\") pod \"iptables-alerter-c9rtw\" (UID: \"9a2cd2a0-f19d-4516-8937-144762938c44\") " pod="openshift-network-operator/iptables-alerter-c9rtw" Jul 02 19:02:45.213099 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.213067 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28hdr\" (UniqueName: \"kubernetes.io/projected/cb60b6fe-625d-4029-9082-cdd088ab2179-kube-api-access-28hdr\") pod \"multus-additional-cni-plugins-9z9qb\" (UID: \"cb60b6fe-625d-4029-9082-cdd088ab2179\") " pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.213346 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.213328 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgswk\" (UniqueName: \"kubernetes.io/projected/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-kube-api-access-tgswk\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:45.213592 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.213576 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9nz8\" (UniqueName: \"kubernetes.io/projected/acfe6cd4-8b9f-4ee3-9473-39afdedf9062-kube-api-access-n9nz8\") pod \"ovnkube-node-kr7sw\" (UID: \"acfe6cd4-8b9f-4ee3-9473-39afdedf9062\") " pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.235384 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.235336 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" event={"ID":"a9ccd326e5056db2ec15b8f22d5755cf","Type":"ContainerStarted","Data":"90f70bfb6db9d86c546cfdbe0e3de8c1c1628e336121ac0b18db80c514d7d1c2"} Jul 02 19:02:45.236400 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.236375 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" event={"ID":"9b6c8b6d42e5b52ab4cf9f706a9be0f7","Type":"ContainerStarted","Data":"165950781c015f118ca299a4a46fa41ca0d595db7919efb703114d4c199f681e"} Jul 02 19:02:45.301587 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.301551 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-sysctl-d\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.301797 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.301596 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-tuned\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.301797 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.301625 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-496gr\" (UniqueName: \"kubernetes.io/projected/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-kube-api-access-496gr\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.301797 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.301672 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:45.301797 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.301701 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-sys-fs\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.301797 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.301726 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-tmp\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.301797 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.301751 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtwv5\" (UniqueName: \"kubernetes.io/projected/3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced-kube-api-access-qtwv5\") pod \"node-ca-rv7zv\" (UID: \"3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced\") " pod="openshift-image-registry/node-ca-rv7zv" Jul 02 19:02:45.301797 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.301769 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-sysctl-d\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302263 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.301879 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:02:45.302263 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.301944 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret podName:9332d088-8ecc-4c56-a0d8-6d35d5acda58 nodeName:}" failed. No retries permitted until 2026-07-02 19:02:45.80192344 +0000 UTC m=+3.093260324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret") pod "global-pull-secret-syncer-8sprk" (UID: "9332d088-8ecc-4c56-a0d8-6d35d5acda58") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:02:45.302263 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302009 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-sys-fs\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.302263 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.301777 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-run\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302263 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302060 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced-host\") pod \"node-ca-rv7zv\" (UID: \"3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced\") " pod="openshift-image-registry/node-ca-rv7zv" Jul 02 19:02:45.302263 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302101 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9332d088-8ecc-4c56-a0d8-6d35d5acda58-dbus\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:45.302263 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302146 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-systemd\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302263 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302160 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced-host\") pod \"node-ca-rv7zv\" (UID: \"3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced\") " pod="openshift-image-registry/node-ca-rv7zv" Jul 02 19:02:45.302263 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302159 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-run\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302263 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302180 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz8ck\" (UniqueName: \"kubernetes.io/projected/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-kube-api-access-qz8ck\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302263 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302220 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9332d088-8ecc-4c56-a0d8-6d35d5acda58-dbus\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302331 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-systemd\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302363 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-modprobe-d\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302493 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-var-lib-kubelet\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302527 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-etc-selinux\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302555 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-modprobe-d\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302586 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-var-lib-kubelet\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302557 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9332d088-8ecc-4c56-a0d8-6d35d5acda58-kubelet-config\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302609 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9332d088-8ecc-4c56-a0d8-6d35d5acda58-kubelet-config\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302628 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-sys\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302638 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-etc-selinux\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302655 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eee2ced5-efa4-498e-83f9-793366d871b7-agent-certs\") pod \"konnectivity-agent-kkggr\" (UID: \"eee2ced5-efa4-498e-83f9-793366d871b7\") " pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302687 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-sys\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302686 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-registration-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302727 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-sysctl-conf\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302731 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-registration-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302754 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.302854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302781 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-lib-modules\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.303756 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302841 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.303756 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302854 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-sysctl-conf\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.303756 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302875 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-lib-modules\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.303756 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.302991 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-host\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.303756 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.303464 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-host\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.303756 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.303528 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-sysconfig\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.303756 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.303631 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-sysconfig\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.303756 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.303679 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-kubernetes\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.303756 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.303753 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-kubernetes\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.304225 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.303945 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-socket-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.304225 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.304176 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-socket-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.304388 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.304230 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-device-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.304388 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.304261 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrlj\" (UniqueName: \"kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj\") pod \"network-check-target-z6zb2\" (UID: \"7c305fe4-c346-4c57-b6c6-1dea6f663801\") " pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:45.304388 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.304341 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-device-dir\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.304542 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.304389 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eee2ced5-efa4-498e-83f9-793366d871b7-konnectivity-ca\") pod \"konnectivity-agent-kkggr\" (UID: \"eee2ced5-efa4-498e-83f9-793366d871b7\") " pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:02:45.304669 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.304414 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced-serviceca\") pod \"node-ca-rv7zv\" (UID: \"3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced\") " pod="openshift-image-registry/node-ca-rv7zv" Jul 02 19:02:45.304884 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.304760 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-tmp\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.305035 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.305008 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eee2ced5-efa4-498e-83f9-793366d871b7-konnectivity-ca\") pod \"konnectivity-agent-kkggr\" (UID: \"eee2ced5-efa4-498e-83f9-793366d871b7\") " pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:02:45.305200 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.305180 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced-serviceca\") pod \"node-ca-rv7zv\" (UID: \"3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced\") " pod="openshift-image-registry/node-ca-rv7zv" Jul 02 19:02:45.305839 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.305815 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-etc-tuned\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.306538 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.306504 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eee2ced5-efa4-498e-83f9-793366d871b7-agent-certs\") pod \"konnectivity-agent-kkggr\" (UID: \"eee2ced5-efa4-498e-83f9-793366d871b7\") " pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:02:45.315344 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.313097 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:02:45.315344 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.313134 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:02:45.315344 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.313168 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wbrlj for pod openshift-network-diagnostics/network-check-target-z6zb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:02:45.315344 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.313285 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj podName:7c305fe4-c346-4c57-b6c6-1dea6f663801 nodeName:}" failed. No retries permitted until 2026-07-02 19:02:45.81323518 +0000 UTC m=+3.104572078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wbrlj" (UniqueName: "kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj") pod "network-check-target-z6zb2" (UID: "7c305fe4-c346-4c57-b6c6-1dea6f663801") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:02:45.318139 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.318109 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtwv5\" (UniqueName: \"kubernetes.io/projected/3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced-kube-api-access-qtwv5\") pod \"node-ca-rv7zv\" (UID: \"3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced\") " pod="openshift-image-registry/node-ca-rv7zv" Jul 02 19:02:45.318256 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.318110 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-496gr\" (UniqueName: \"kubernetes.io/projected/85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20-kube-api-access-496gr\") pod \"aws-ebs-csi-driver-node-7q6rc\" (UID: \"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.318689 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.318659 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz8ck\" (UniqueName: \"kubernetes.io/projected/0fdcaf03-1c45-4755-804b-5dfad0a3f3a9-kube-api-access-qz8ck\") pod \"tuned-ckk29\" (UID: \"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9\") " pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.389652 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.389545 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:02:45.396340 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.396303 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v2xxk" Jul 02 19:02:45.405579 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.405548 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7tb99" Jul 02 19:02:45.405764 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:45.405736 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacfe6cd4_8b9f_4ee3_9473_39afdedf9062.slice/crio-6e94554b57bebcdba9c5e0f5edb0a4285758c788847ffb31077b120f08b7fdd5 WatchSource:0}: Error finding container 6e94554b57bebcdba9c5e0f5edb0a4285758c788847ffb31077b120f08b7fdd5: Status 404 returned error can't find the container with id 6e94554b57bebcdba9c5e0f5edb0a4285758c788847ffb31077b120f08b7fdd5 Jul 02 19:02:45.410293 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.410268 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" Jul 02 19:02:45.410842 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:45.410808 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe8227ad_efbc_48d6_bf5b_21902eeeb40f.slice/crio-397a1797b581c0a94561f61a00e945c4d95726246e0cb04a309191d2dd251653 WatchSource:0}: Error finding container 397a1797b581c0a94561f61a00e945c4d95726246e0cb04a309191d2dd251653: Status 404 returned error can't find the container with id 397a1797b581c0a94561f61a00e945c4d95726246e0cb04a309191d2dd251653 Jul 02 19:02:45.417148 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.417122 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c9rtw" Jul 02 19:02:45.418755 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:45.418725 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45980b15_071c_4aa7_a6e6_8ae7c9d48de2.slice/crio-4e3564627b1b93d04ffc7fbbec8b461375cba776c482eee7bce413addfd65797 WatchSource:0}: Error finding container 4e3564627b1b93d04ffc7fbbec8b461375cba776c482eee7bce413addfd65797: Status 404 returned error can't find the container with id 4e3564627b1b93d04ffc7fbbec8b461375cba776c482eee7bce413addfd65797 Jul 02 19:02:45.422517 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.422497 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" Jul 02 19:02:45.422845 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:45.422822 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb60b6fe_625d_4029_9082_cdd088ab2179.slice/crio-52198bd8313dcc5f41f4cd52d140c902593332b9ab1eb8f8b50b2f77d8f53dbd WatchSource:0}: Error finding container 52198bd8313dcc5f41f4cd52d140c902593332b9ab1eb8f8b50b2f77d8f53dbd: Status 404 returned error can't find the container with id 52198bd8313dcc5f41f4cd52d140c902593332b9ab1eb8f8b50b2f77d8f53dbd Jul 02 19:02:45.429404 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.429378 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:02:45.429498 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:45.429459 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a2cd2a0_f19d_4516_8937_144762938c44.slice/crio-56c3abe88269178252b7cee1201366535ff92a67ad1ddfd57c087369f08502e0 WatchSource:0}: Error finding container 56c3abe88269178252b7cee1201366535ff92a67ad1ddfd57c087369f08502e0: Status 404 returned error can't find the container with id 56c3abe88269178252b7cee1201366535ff92a67ad1ddfd57c087369f08502e0 Jul 02 19:02:45.434403 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.434383 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ckk29" Jul 02 19:02:45.434971 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:45.434942 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85cf5bcd_aaa6_4c93_ac0d_3161ee5cff20.slice/crio-0a7ab2922fbe55a200807532a04049f4e82b6c4af14a8a4078ae69ed19c71057 WatchSource:0}: Error finding container 0a7ab2922fbe55a200807532a04049f4e82b6c4af14a8a4078ae69ed19c71057: Status 404 returned error can't find the container with id 0a7ab2922fbe55a200807532a04049f4e82b6c4af14a8a4078ae69ed19c71057 Jul 02 19:02:45.439211 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.439192 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rv7zv" Jul 02 19:02:45.442591 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:45.442559 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee2ced5_efa4_498e_83f9_793366d871b7.slice/crio-25b8526920782b0e9e523a5b3db906fc1eacc8f5998bc605998b249d8afd14a6 WatchSource:0}: Error finding container 25b8526920782b0e9e523a5b3db906fc1eacc8f5998bc605998b249d8afd14a6: Status 404 returned error can't find the container with id 25b8526920782b0e9e523a5b3db906fc1eacc8f5998bc605998b249d8afd14a6 Jul 02 19:02:45.449026 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:45.448996 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fdcaf03_1c45_4755_804b_5dfad0a3f3a9.slice/crio-f043454baf8fc9f08d3e7244a1db66bab3294e2d42298812396f95b522c10a4f WatchSource:0}: Error finding container f043454baf8fc9f08d3e7244a1db66bab3294e2d42298812396f95b522c10a4f: Status 404 returned error can't find the container with id f043454baf8fc9f08d3e7244a1db66bab3294e2d42298812396f95b522c10a4f Jul 02 19:02:45.453129 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:02:45.453081 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb8b0ed_ff24_4cdb_8bdc_288cc6a8aced.slice/crio-a6e3ec979b62471ecf39b3f3e80d5561409c89b1a4a9616ae0811a46f0c06ce6 WatchSource:0}: Error finding container a6e3ec979b62471ecf39b3f3e80d5561409c89b1a4a9616ae0811a46f0c06ce6: Status 404 returned error can't find the container with id a6e3ec979b62471ecf39b3f3e80d5561409c89b1a4a9616ae0811a46f0c06ce6 Jul 02 19:02:45.544604 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.544561 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:02:45.708673 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.708576 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:45.708840 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.708778 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:02:45.708890 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.708866 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs podName:3afa8f46-c8f3-4b0b-a33e-42d05fefc58f nodeName:}" failed. No retries permitted until 2026-07-02 19:02:46.708844688 +0000 UTC m=+4.000181574 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs") pod "network-metrics-daemon-27pfj" (UID: "3afa8f46-c8f3-4b0b-a33e-42d05fefc58f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:02:45.809755 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.809618 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:45.809938 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.809809 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:02:45.809938 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.809896 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret podName:9332d088-8ecc-4c56-a0d8-6d35d5acda58 nodeName:}" failed. No retries permitted until 2026-07-02 19:02:46.809871893 +0000 UTC m=+4.101208793 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret") pod "global-pull-secret-syncer-8sprk" (UID: "9332d088-8ecc-4c56-a0d8-6d35d5acda58") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:02:45.911052 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:45.911010 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrlj\" (UniqueName: \"kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj\") pod \"network-check-target-z6zb2\" (UID: \"7c305fe4-c346-4c57-b6c6-1dea6f663801\") " pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:45.911285 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.911210 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:02:45.911285 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.911235 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:02:45.911285 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.911250 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wbrlj for pod openshift-network-diagnostics/network-check-target-z6zb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:02:45.911465 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:45.911319 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj podName:7c305fe4-c346-4c57-b6c6-1dea6f663801 nodeName:}" failed. No retries permitted until 2026-07-02 19:02:46.911299482 +0000 UTC m=+4.202636376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbrlj" (UniqueName: "kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj") pod "network-check-target-z6zb2" (UID: "7c305fe4-c346-4c57-b6c6-1dea6f663801") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:02:46.128491 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.128307 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 18:57:45 +0000 UTC" deadline="2028-03-03 14:36:33.347364265 +0000 UTC" Jul 02 19:02:46.128491 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.128381 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14635h33m47.218987618s" Jul 02 19:02:46.240645 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.240602 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rv7zv" event={"ID":"3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced","Type":"ContainerStarted","Data":"a6e3ec979b62471ecf39b3f3e80d5561409c89b1a4a9616ae0811a46f0c06ce6"} Jul 02 19:02:46.242059 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.242026 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kkggr" event={"ID":"eee2ced5-efa4-498e-83f9-793366d871b7","Type":"ContainerStarted","Data":"25b8526920782b0e9e523a5b3db906fc1eacc8f5998bc605998b249d8afd14a6"} Jul 02 19:02:46.243442 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.243394 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" event={"ID":"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20","Type":"ContainerStarted","Data":"0a7ab2922fbe55a200807532a04049f4e82b6c4af14a8a4078ae69ed19c71057"} Jul 02 19:02:46.244733 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.244703 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" event={"ID":"cb60b6fe-625d-4029-9082-cdd088ab2179","Type":"ContainerStarted","Data":"52198bd8313dcc5f41f4cd52d140c902593332b9ab1eb8f8b50b2f77d8f53dbd"} Jul 02 19:02:46.245948 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.245919 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v2xxk" event={"ID":"fe8227ad-efbc-48d6-bf5b-21902eeeb40f","Type":"ContainerStarted","Data":"397a1797b581c0a94561f61a00e945c4d95726246e0cb04a309191d2dd251653"} Jul 02 19:02:46.248221 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.248194 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ckk29" event={"ID":"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9","Type":"ContainerStarted","Data":"f043454baf8fc9f08d3e7244a1db66bab3294e2d42298812396f95b522c10a4f"} Jul 02 19:02:46.249894 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.249862 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c9rtw" event={"ID":"9a2cd2a0-f19d-4516-8937-144762938c44","Type":"ContainerStarted","Data":"56c3abe88269178252b7cee1201366535ff92a67ad1ddfd57c087369f08502e0"} Jul 02 19:02:46.251544 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.251518 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7tb99" event={"ID":"45980b15-071c-4aa7-a6e6-8ae7c9d48de2","Type":"ContainerStarted","Data":"4e3564627b1b93d04ffc7fbbec8b461375cba776c482eee7bce413addfd65797"} Jul 02 19:02:46.252733 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.252708 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" event={"ID":"acfe6cd4-8b9f-4ee3-9473-39afdedf9062","Type":"ContainerStarted","Data":"6e94554b57bebcdba9c5e0f5edb0a4285758c788847ffb31077b120f08b7fdd5"} Jul 02 19:02:46.718008 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.717964 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:46.718255 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:46.718197 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:02:46.718342 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:46.718281 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs podName:3afa8f46-c8f3-4b0b-a33e-42d05fefc58f nodeName:}" failed. No retries permitted until 2026-07-02 19:02:48.718254715 +0000 UTC m=+6.009591612 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs") pod "network-metrics-daemon-27pfj" (UID: "3afa8f46-c8f3-4b0b-a33e-42d05fefc58f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:02:46.818454 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.818415 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:46.818657 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:46.818602 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:02:46.818726 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:46.818691 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret podName:9332d088-8ecc-4c56-a0d8-6d35d5acda58 nodeName:}" failed. No retries permitted until 2026-07-02 19:02:48.81866682 +0000 UTC m=+6.110003716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret") pod "global-pull-secret-syncer-8sprk" (UID: "9332d088-8ecc-4c56-a0d8-6d35d5acda58") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:02:46.919709 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:46.919663 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrlj\" (UniqueName: \"kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj\") pod \"network-check-target-z6zb2\" (UID: \"7c305fe4-c346-4c57-b6c6-1dea6f663801\") " pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:46.919912 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:46.919900 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:02:46.919983 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:46.919924 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:02:46.919983 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:46.919938 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wbrlj for pod openshift-network-diagnostics/network-check-target-z6zb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:02:46.920099 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:46.920004 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj podName:7c305fe4-c346-4c57-b6c6-1dea6f663801 nodeName:}" failed. No retries permitted until 2026-07-02 19:02:48.919983428 +0000 UTC m=+6.211320322 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbrlj" (UniqueName: "kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj") pod "network-check-target-z6zb2" (UID: "7c305fe4-c346-4c57-b6c6-1dea6f663801") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:02:47.129166 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:47.129117 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 18:57:45 +0000 UTC" deadline="2027-12-03 02:05:35.347585961 +0000 UTC" Jul 02 19:02:47.129166 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:47.129164 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12439h2m48.218427988s" Jul 02 19:02:47.234620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:47.234564 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:47.234794 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:47.234748 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:02:47.235117 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:47.234899 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:47.235117 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:47.234931 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:47.235117 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:47.235012 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:02:47.235117 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:47.235102 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:02:48.737625 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:48.737592 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:48.737984 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:48.737752 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:02:48.737984 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:48.737809 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs podName:3afa8f46-c8f3-4b0b-a33e-42d05fefc58f nodeName:}" failed. No retries permitted until 2026-07-02 19:02:52.737791832 +0000 UTC m=+10.029128715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs") pod "network-metrics-daemon-27pfj" (UID: "3afa8f46-c8f3-4b0b-a33e-42d05fefc58f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:02:48.838731 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:48.838173 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:48.838731 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:48.838313 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:02:48.838731 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:48.838378 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret podName:9332d088-8ecc-4c56-a0d8-6d35d5acda58 nodeName:}" failed. No retries permitted until 2026-07-02 19:02:52.838359709 +0000 UTC m=+10.129696610 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret") pod "global-pull-secret-syncer-8sprk" (UID: "9332d088-8ecc-4c56-a0d8-6d35d5acda58") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:02:48.939237 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:48.939025 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrlj\" (UniqueName: \"kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj\") pod \"network-check-target-z6zb2\" (UID: \"7c305fe4-c346-4c57-b6c6-1dea6f663801\") " pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:48.944042 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:48.940496 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:02:48.944042 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:48.940524 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:02:48.944042 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:48.940547 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wbrlj for pod openshift-network-diagnostics/network-check-target-z6zb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:02:48.944042 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:48.940631 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj podName:7c305fe4-c346-4c57-b6c6-1dea6f663801 nodeName:}" failed. No retries permitted until 2026-07-02 19:02:52.940602139 +0000 UTC m=+10.231939032 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbrlj" (UniqueName: "kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj") pod "network-check-target-z6zb2" (UID: "7c305fe4-c346-4c57-b6c6-1dea6f663801") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:02:49.232457 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:49.231858 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:49.232457 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:49.232002 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:02:49.232457 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:49.232233 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:49.232457 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:49.232319 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:02:49.232868 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:49.232463 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:49.232868 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:49.232539 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:02:51.232970 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:51.232887 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:51.233421 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:51.232897 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:51.233421 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:51.233029 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:02:51.233421 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:51.233131 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:02:51.233421 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:51.233179 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:51.233421 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:51.233243 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:02:52.773688 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:52.773148 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:52.773688 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:52.773310 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:02:52.773688 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:52.773375 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs podName:3afa8f46-c8f3-4b0b-a33e-42d05fefc58f nodeName:}" failed. No retries permitted until 2026-07-02 19:03:00.773355231 +0000 UTC m=+18.064692117 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs") pod "network-metrics-daemon-27pfj" (UID: "3afa8f46-c8f3-4b0b-a33e-42d05fefc58f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:02:52.874460 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:52.874418 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:52.874627 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:52.874588 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:02:52.874680 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:52.874667 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret podName:9332d088-8ecc-4c56-a0d8-6d35d5acda58 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:00.874645824 +0000 UTC m=+18.165982717 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret") pod "global-pull-secret-syncer-8sprk" (UID: "9332d088-8ecc-4c56-a0d8-6d35d5acda58") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:02:52.975690 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:52.975646 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrlj\" (UniqueName: \"kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj\") pod \"network-check-target-z6zb2\" (UID: \"7c305fe4-c346-4c57-b6c6-1dea6f663801\") " pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:52.975875 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:52.975852 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:02:52.975925 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:52.975882 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:02:52.975925 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:52.975896 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wbrlj for pod openshift-network-diagnostics/network-check-target-z6zb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:02:52.975991 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:52.975958 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj podName:7c305fe4-c346-4c57-b6c6-1dea6f663801 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:00.975939678 +0000 UTC m=+18.267276562 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbrlj" (UniqueName: "kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj") pod "network-check-target-z6zb2" (UID: "7c305fe4-c346-4c57-b6c6-1dea6f663801") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:02:53.233712 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:53.233012 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:53.233712 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:53.233149 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:02:53.233712 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:53.233552 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:53.233712 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:53.233641 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:02:53.233712 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:53.233692 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:53.233999 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:53.233755 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:02:55.232576 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:55.232471 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:55.233270 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:55.233147 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:02:55.233270 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:55.232471 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:55.233451 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:55.233368 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:02:55.233915 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:55.233759 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:55.233915 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:55.233878 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:02:57.231523 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:57.231266 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:57.231950 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:57.231295 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:57.231950 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:57.231266 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:57.231950 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:57.231773 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:02:57.231950 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:57.231622 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:02:57.231950 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:57.231873 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:02:57.277401 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:57.277355 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" event={"ID":"9b6c8b6d42e5b52ab4cf9f706a9be0f7","Type":"ContainerStarted","Data":"48dd0c276a5488b41c5cf4e348e1aa10686ab7ac46c4d9cd1d89ed8d2cf60392"} Jul 02 19:02:57.290521 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:57.290471 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-101.ec2.internal" podStartSLOduration=13.290453918 podStartE2EDuration="13.290453918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:02:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:02:57.29043635 +0000 UTC m=+14.581773251" watchObservedRunningTime="2026-07-02 19:02:57.290453918 +0000 UTC m=+14.581790818" Jul 02 19:02:58.280873 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:58.280831 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" event={"ID":"a9ccd326e5056db2ec15b8f22d5755cf","Type":"ContainerStarted","Data":"81a3f18ce1af5683a04181e4c931fc7e77c3d4ab030d84423100f395ea14d91b"} Jul 02 19:02:58.282431 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:58.282403 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rv7zv" event={"ID":"3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced","Type":"ContainerStarted","Data":"0d8c5f5641c318bebace1b2e2f953dcb8358410ae362f91c29e7ed0bca8fb5a6"} Jul 02 19:02:58.283979 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:58.283951 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kkggr" event={"ID":"eee2ced5-efa4-498e-83f9-793366d871b7","Type":"ContainerStarted","Data":"0fc4d691d3a209b79c08fc0875137417bd7c8c84a745d7f48c274a66cb41d7de"} Jul 02 19:02:58.285810 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:58.285789 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" event={"ID":"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20","Type":"ContainerStarted","Data":"d3c59c6c19f2dae1c5ea29563b5c8a136c574cb8f973aca37e0e39662cf4a6fa"} Jul 02 19:02:58.287454 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:58.287423 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" event={"ID":"cb60b6fe-625d-4029-9082-cdd088ab2179","Type":"ContainerStarted","Data":"be8da988f7c8f24b07baf9a7f097e58b36dced913ac31cd181a676d5575908aa"} Jul 02 19:02:58.288981 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:58.288953 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v2xxk" event={"ID":"fe8227ad-efbc-48d6-bf5b-21902eeeb40f","Type":"ContainerStarted","Data":"bb7f162ebc2b374eea3570ddaa35a01ff72f549672e4ed1bd9406239760fd6ce"} Jul 02 19:02:58.290566 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:58.290544 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c9rtw" event={"ID":"9a2cd2a0-f19d-4516-8937-144762938c44","Type":"ContainerStarted","Data":"4d74e4fe506421fb3f2d4aa7ae1ad2d8f36b2ac2769d34aaf9fbfa5415cb7938"} Jul 02 19:02:58.307837 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:58.307789 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rv7zv" podStartSLOduration=4.501428871 podStartE2EDuration="15.307769163s" totalImagesPullingTime="10.806340292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:02:58.307652672 +0000 UTC m=+15.598989574" watchObservedRunningTime="2026-07-02 19:02:58.307769163 +0000 UTC m=+15.599106067" Jul 02 19:02:58.322377 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:58.322320 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-v2xxk" podStartSLOduration=4.475278579 podStartE2EDuration="15.322298078s" totalImagesPullingTime="10.847019499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:02:58.321658603 +0000 UTC m=+15.612995504" watchObservedRunningTime="2026-07-02 19:02:58.322298078 +0000 UTC m=+15.613634979" Jul 02 19:02:58.354686 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:58.354632 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c9rtw" podStartSLOduration=4.52093895 podStartE2EDuration="15.354614477s" totalImagesPullingTime="10.833675527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:02:58.354338801 +0000 UTC m=+15.645675701" watchObservedRunningTime="2026-07-02 19:02:58.354614477 +0000 UTC m=+15.645951379" Jul 02 19:02:58.368434 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:58.368379 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kkggr" podStartSLOduration=4.502715061 podStartE2EDuration="15.368360625s" totalImagesPullingTime="10.865645564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:02:58.367769129 +0000 UTC m=+15.659106030" watchObservedRunningTime="2026-07-02 19:02:58.368360625 +0000 UTC m=+15.659697516" Jul 02 19:02:59.125176 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:59.125133 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:02:59.125715 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:59.125699 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:02:59.231613 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:59.231579 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:02:59.231795 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:59.231579 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:02:59.231795 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:59.231708 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:02:59.231900 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:59.231816 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:02:59.231900 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:02:59.231590 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:02:59.231982 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:02:59.231901 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:00.294373 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:00.294345 2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 02 19:03:00.831993 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:00.831775 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:00.832179 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:00.831923 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:03:00.832179 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:00.832109 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs podName:3afa8f46-c8f3-4b0b-a33e-42d05fefc58f nodeName:}" failed. No retries permitted until 2026-07-02 19:03:16.832088075 +0000 UTC m=+34.123424982 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs") pod "network-metrics-daemon-27pfj" (UID: "3afa8f46-c8f3-4b0b-a33e-42d05fefc58f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:03:00.933314 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:00.933226 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:00.933464 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:00.933397 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:03:00.933503 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:00.933474 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret podName:9332d088-8ecc-4c56-a0d8-6d35d5acda58 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:16.933457663 +0000 UTC m=+34.224794557 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret") pod "global-pull-secret-syncer-8sprk" (UID: "9332d088-8ecc-4c56-a0d8-6d35d5acda58") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:03:01.034257 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:01.034218 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrlj\" (UniqueName: \"kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj\") pod \"network-check-target-z6zb2\" (UID: \"7c305fe4-c346-4c57-b6c6-1dea6f663801\") " pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:01.034439 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:01.034412 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:03:01.034439 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:01.034437 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:03:01.034547 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:01.034450 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wbrlj for pod openshift-network-diagnostics/network-check-target-z6zb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:03:01.034547 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:01.034520 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj podName:7c305fe4-c346-4c57-b6c6-1dea6f663801 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:17.034500623 +0000 UTC m=+34.325837518 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbrlj" (UniqueName: "kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj") pod "network-check-target-z6zb2" (UID: "7c305fe4-c346-4c57-b6c6-1dea6f663801") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:03:01.231860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:01.231636 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:01.231860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:01.231676 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:01.231860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:01.231636 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:01.231860 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:01.231782 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:01.232185 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:01.231935 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:01.232185 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:01.232027 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:01.298222 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:01.298189 2571 generic.go:356] "Generic (PLEG): container finished" podID="a9ccd326e5056db2ec15b8f22d5755cf" containerID="81a3f18ce1af5683a04181e4c931fc7e77c3d4ab030d84423100f395ea14d91b" exitCode=0 Jul 02 19:03:01.298638 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:01.298265 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" event={"ID":"a9ccd326e5056db2ec15b8f22d5755cf","Type":"ContainerDied","Data":"81a3f18ce1af5683a04181e4c931fc7e77c3d4ab030d84423100f395ea14d91b"} Jul 02 19:03:01.300084 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:01.300049 2571 generic.go:356] "Generic (PLEG): container finished" podID="cb60b6fe-625d-4029-9082-cdd088ab2179" containerID="be8da988f7c8f24b07baf9a7f097e58b36dced913ac31cd181a676d5575908aa" exitCode=0 Jul 02 19:03:01.300188 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:01.300104 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" event={"ID":"cb60b6fe-625d-4029-9082-cdd088ab2179","Type":"ContainerDied","Data":"be8da988f7c8f24b07baf9a7f097e58b36dced913ac31cd181a676d5575908aa"} Jul 02 19:03:03.232179 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:03.232134 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:03.232179 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:03.232161 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:03.232720 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:03.232295 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:03.232720 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:03.232344 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:03.232720 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:03.232431 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:03.232720 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:03.232516 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:05.231416 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:05.231335 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:05.231416 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:05.231340 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:05.231969 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:05.231527 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:05.231969 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:05.231801 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:05.231969 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:05.231910 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:05.232095 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:05.232006 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:05.723356 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:05.723107 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 19:03:06.165389 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.165288 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T19:03:05.723348261Z","UUID":"81f904cd-6178-4b16-b592-a7483b38803b","Handler":null,"Name":"","Endpoint":""} Jul 02 19:03:06.166711 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.166688 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 19:03:06.166857 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.166729 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 19:03:06.312198 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.312120 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" event={"ID":"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20","Type":"ContainerStarted","Data":"5e064f0ab2813592ad04047491a1b1c6e74d39846134481e613bad7aa92efd57"} Jul 02 19:03:06.314041 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.314001 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ckk29" event={"ID":"0fdcaf03-1c45-4755-804b-5dfad0a3f3a9","Type":"ContainerStarted","Data":"8f3aa60114b23f36811e922e92778078cca9292d6c484ee2a1314cb8867109c9"} Jul 02 19:03:06.316291 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.315991 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7tb99" event={"ID":"45980b15-071c-4aa7-a6e6-8ae7c9d48de2","Type":"ContainerStarted","Data":"1bcabae24617d0e9e0279d37d0f33e4dc5a8042e595e66b207381aa28900fbf6"} Jul 02 19:03:06.320310 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.320285 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" event={"ID":"acfe6cd4-8b9f-4ee3-9473-39afdedf9062","Type":"ContainerStarted","Data":"949c5c7d175ebe002b32a42c364b56de65e480ce1a13bc2aa1f0c55feede09a1"} Jul 02 19:03:06.320449 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.320364 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" event={"ID":"acfe6cd4-8b9f-4ee3-9473-39afdedf9062","Type":"ContainerStarted","Data":"b02140a89521723adf7e43245665e0aad910bdbaef743ee9cd83b34c3cf5f16f"} Jul 02 19:03:06.320449 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.320413 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" event={"ID":"acfe6cd4-8b9f-4ee3-9473-39afdedf9062","Type":"ContainerStarted","Data":"f25367c7eec850ab789efa4b70257ebff54144e2fb52c4403cf4943e1268bbc2"} Jul 02 19:03:06.320578 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.320454 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" event={"ID":"acfe6cd4-8b9f-4ee3-9473-39afdedf9062","Type":"ContainerStarted","Data":"b3f3146a112574482fce740d089e7839ac4aab77f86d3593f33374b9c1dbba4f"} Jul 02 19:03:06.320578 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.320501 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" event={"ID":"acfe6cd4-8b9f-4ee3-9473-39afdedf9062","Type":"ContainerStarted","Data":"4293f389270aa9663ac116728c2c6745f990f2bdb39bc0a0e68b62c0f5b061ac"} Jul 02 19:03:06.320578 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.320546 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" event={"ID":"acfe6cd4-8b9f-4ee3-9473-39afdedf9062","Type":"ContainerStarted","Data":"5bdfe9037603aee363e3c4d1ad01abc3bfee56a9b3af4011e3550fe00298e8ff"} Jul 02 19:03:06.322382 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.322359 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" event={"ID":"a9ccd326e5056db2ec15b8f22d5755cf","Type":"ContainerStarted","Data":"f1a50d87497db4cb88c8a2b144f2cc219d21d5eebb54c4a1fb3c266216581a51"} Jul 02 19:03:06.356397 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.356335 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7tb99" podStartSLOduration=3.487716119 podStartE2EDuration="23.35631486s" totalImagesPullingTime="19.868598741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:06.356217222 +0000 UTC m=+23.647554123" watchObservedRunningTime="2026-07-02 19:03:06.35631486 +0000 UTC m=+23.647651762" Jul 02 19:03:06.356975 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.356947 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ckk29" podStartSLOduration=12.271919683 podStartE2EDuration="23.35693944s" totalImagesPullingTime="11.085019757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:06.334408353 +0000 UTC m=+23.625745256" watchObservedRunningTime="2026-07-02 19:03:06.35693944 +0000 UTC m=+23.648276342" Jul 02 19:03:06.371886 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.371826 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-101.ec2.internal" podStartSLOduration=22.371804729 podStartE2EDuration="22.371804729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:02:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:06.371644339 +0000 UTC m=+23.662981241" watchObservedRunningTime="2026-07-02 19:03:06.371804729 +0000 UTC m=+23.663141631" Jul 02 19:03:06.539142 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:06.539117 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v2xxk_fe8227ad-efbc-48d6-bf5b-21902eeeb40f/dns-node-resolver/0.log" Jul 02 19:03:07.232328 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:07.232110 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:07.232547 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:07.232111 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:07.232547 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:07.232354 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:07.232547 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:07.232121 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:07.232547 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:07.232458 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:07.232547 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:07.232520 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:07.327823 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:07.327735 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" event={"ID":"85cf5bcd-aaa6-4c93-ac0d-3161ee5cff20","Type":"ContainerStarted","Data":"4b01510f3545d9e44ca7ef905b85763c0d3db8a40254a11ed327704a97ceca25"} Jul 02 19:03:07.525622 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:07.525600 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rv7zv_3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced/node-ca/0.log" Jul 02 19:03:08.333235 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:08.333200 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" event={"ID":"acfe6cd4-8b9f-4ee3-9473-39afdedf9062","Type":"ContainerStarted","Data":"5d943362ed6064c9f89dc9fed9e237da79523205660831afebe34092bbd20b01"} Jul 02 19:03:09.231662 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:09.231626 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:09.231851 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:09.231626 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:09.231851 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:09.231733 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:09.231851 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:09.231630 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:09.231851 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:09.231800 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:09.232061 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:09.231874 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:11.231674 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:11.231590 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:11.232088 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:11.231593 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:11.232088 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:11.231719 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:11.232088 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:11.231843 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:11.232088 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:11.231590 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:11.232088 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:11.231952 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:12.344388 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:12.344209 2571 generic.go:356] "Generic (PLEG): container finished" podID="cb60b6fe-625d-4029-9082-cdd088ab2179" containerID="6310e1098de9b93ef5d75c3c0b48c8e48dcae05a16c76fe6f64ba3c412edbe82" exitCode=0 Jul 02 19:03:12.345171 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:12.344276 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" event={"ID":"cb60b6fe-625d-4029-9082-cdd088ab2179","Type":"ContainerDied","Data":"6310e1098de9b93ef5d75c3c0b48c8e48dcae05a16c76fe6f64ba3c412edbe82"} Jul 02 19:03:12.348108 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:12.348084 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" event={"ID":"acfe6cd4-8b9f-4ee3-9473-39afdedf9062","Type":"ContainerStarted","Data":"b8d9849e9dd7336e06c28a407c0097374723431ba84eb8a8fb2438ff7a45175e"} Jul 02 19:03:12.348628 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:12.348453 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:03:12.348628 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:12.348526 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:03:12.348628 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:12.348576 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:03:12.363567 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:12.363542 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:03:12.363704 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:12.363648 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:03:12.369435 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:12.369397 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7q6rc" podStartSLOduration=9.594360625 podStartE2EDuration="29.369381071s" totalImagesPullingTime="19.775020446s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:07.357886046 +0000 UTC m=+24.649222946" watchObservedRunningTime="2026-07-02 19:03:12.369381071 +0000 UTC m=+29.660717981" Jul 02 19:03:12.395954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:12.395902 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" podStartSLOduration=9.449880405 podStartE2EDuration="29.395886792s" totalImagesPullingTime="19.946006387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:12.394549531 +0000 UTC m=+29.685886431" watchObservedRunningTime="2026-07-02 19:03:12.395886792 +0000 UTC m=+29.687223691" Jul 02 19:03:13.233237 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:13.233191 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:13.233391 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:13.233224 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:13.233391 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:13.233336 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:13.233391 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:13.233346 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:13.233535 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:13.233426 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:13.233535 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:13.233464 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:13.352083 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:13.352038 2571 generic.go:356] "Generic (PLEG): container finished" podID="cb60b6fe-625d-4029-9082-cdd088ab2179" containerID="72c166f0d996742a3b18bf256ffb0b00e0c852ec61be3521abd893321756a463" exitCode=0 Jul 02 19:03:13.352443 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:13.352118 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" event={"ID":"cb60b6fe-625d-4029-9082-cdd088ab2179","Type":"ContainerDied","Data":"72c166f0d996742a3b18bf256ffb0b00e0c852ec61be3521abd893321756a463"} Jul 02 19:03:14.252686 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:14.252653 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:03:14.252804 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:14.252772 2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 02 19:03:14.253150 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:14.253128 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kkggr" Jul 02 19:03:14.356103 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:14.356055 2571 generic.go:356] "Generic (PLEG): container finished" podID="cb60b6fe-625d-4029-9082-cdd088ab2179" containerID="57a368ffc6cce99c9afaf2298531e0c9118ab906c164f10314922dd163af7111" exitCode=0 Jul 02 19:03:14.356486 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:14.356127 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" event={"ID":"cb60b6fe-625d-4029-9082-cdd088ab2179","Type":"ContainerDied","Data":"57a368ffc6cce99c9afaf2298531e0c9118ab906c164f10314922dd163af7111"} Jul 02 19:03:15.231716 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:15.231682 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:15.231866 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:15.231723 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:15.231866 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:15.231691 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:15.231866 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:15.231795 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:15.231973 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:15.231879 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:15.231973 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:15.231962 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:16.857183 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:16.857152 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:16.857555 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:16.857290 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:03:16.857555 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:16.857347 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs podName:3afa8f46-c8f3-4b0b-a33e-42d05fefc58f nodeName:}" failed. No retries permitted until 2026-07-02 19:03:48.85733283 +0000 UTC m=+66.148669711 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs") pod "network-metrics-daemon-27pfj" (UID: "3afa8f46-c8f3-4b0b-a33e-42d05fefc58f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:03:16.957963 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:16.957929 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:16.958162 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:16.958097 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:03:16.958162 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:16.958158 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret podName:9332d088-8ecc-4c56-a0d8-6d35d5acda58 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:48.958143442 +0000 UTC m=+66.249480320 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret") pod "global-pull-secret-syncer-8sprk" (UID: "9332d088-8ecc-4c56-a0d8-6d35d5acda58") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:03:17.059272 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:17.059237 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrlj\" (UniqueName: \"kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj\") pod \"network-check-target-z6zb2\" (UID: \"7c305fe4-c346-4c57-b6c6-1dea6f663801\") " pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:17.059427 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:17.059405 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:03:17.059427 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:17.059423 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:03:17.059535 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:17.059432 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wbrlj for pod openshift-network-diagnostics/network-check-target-z6zb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:03:17.059535 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:17.059494 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj podName:7c305fe4-c346-4c57-b6c6-1dea6f663801 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:49.059476547 +0000 UTC m=+66.350813426 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbrlj" (UniqueName: "kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj") pod "network-check-target-z6zb2" (UID: "7c305fe4-c346-4c57-b6c6-1dea6f663801") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:03:17.231516 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:17.231431 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:17.231668 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:17.231431 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:17.231668 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:17.231567 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:17.231668 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:17.231642 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:17.231668 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:17.231431 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:17.231923 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:17.231741 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:19.231366 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:19.231325 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:19.231366 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:19.231351 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:19.231974 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:19.231335 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:19.231974 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:19.231495 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:19.231974 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:19.231602 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:19.231974 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:19.231705 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:20.372025 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:20.371991 2571 generic.go:356] "Generic (PLEG): container finished" podID="cb60b6fe-625d-4029-9082-cdd088ab2179" containerID="046a7e711472a54095928740a4bcfa51367f44834f7c3c18a149a46fc5c6bfb6" exitCode=0 Jul 02 19:03:20.372437 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:20.372051 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" event={"ID":"cb60b6fe-625d-4029-9082-cdd088ab2179","Type":"ContainerDied","Data":"046a7e711472a54095928740a4bcfa51367f44834f7c3c18a149a46fc5c6bfb6"} Jul 02 19:03:21.231431 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:21.231394 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:21.231628 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:21.231393 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:21.231628 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:21.231531 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:21.231628 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:21.231396 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:21.231628 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:21.231599 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:21.231779 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:21.231647 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:21.376629 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:21.376592 2571 generic.go:356] "Generic (PLEG): container finished" podID="cb60b6fe-625d-4029-9082-cdd088ab2179" containerID="2cae07d9cc9d79281d2105881085a44c44e720dda3ddb62012b454952f094cfc" exitCode=0 Jul 02 19:03:21.377033 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:21.376651 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" event={"ID":"cb60b6fe-625d-4029-9082-cdd088ab2179","Type":"ContainerDied","Data":"2cae07d9cc9d79281d2105881085a44c44e720dda3ddb62012b454952f094cfc"} Jul 02 19:03:22.381867 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:22.381830 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" event={"ID":"cb60b6fe-625d-4029-9082-cdd088ab2179","Type":"ContainerStarted","Data":"971f2a283c496b7bcbdc907f8b24a4a98b1fec811b8f7bae56062c19eb628edd"} Jul 02 19:03:22.404516 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:22.404467 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9z9qb" podStartSLOduration=10.861344946 podStartE2EDuration="39.404452498s" totalImagesPullingTime="28.543107552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:22.404136004 +0000 UTC m=+39.695472905" watchObservedRunningTime="2026-07-02 19:03:22.404452498 +0000 UTC m=+39.695789398" Jul 02 19:03:23.232590 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:23.232553 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:23.232777 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:23.232644 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:23.232777 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:23.232654 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:23.232777 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:23.232682 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:23.232777 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:23.232747 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:23.232943 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:23.232812 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:25.231883 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:25.231848 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:25.231883 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:25.231871 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:25.232342 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:25.231871 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:25.232342 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:25.231990 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:25.232342 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:25.232134 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:25.232342 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:25.232194 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:27.231134 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:27.231101 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:27.231526 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:27.231101 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:27.231526 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:27.231230 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:27.231526 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:27.231101 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:27.231526 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:27.231283 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:27.231526 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:27.231341 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:27.835035 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:27.834809 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8sprk"] Jul 02 19:03:27.835191 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:27.835152 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:27.835275 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:27.835250 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:27.835535 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:27.835513 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-27pfj"] Jul 02 19:03:27.835617 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:27.835605 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:27.835703 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:27.835686 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:27.849655 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:27.849626 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z6zb2"] Jul 02 19:03:27.849785 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:27.849717 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:27.849827 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:27.849793 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:29.231640 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:29.231609 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:29.231640 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:29.231629 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:29.232279 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:29.231609 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:29.232279 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:29.231735 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:29.232279 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:29.231799 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:29.232279 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:29.231851 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:31.231408 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:31.231370 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:31.232003 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:31.231382 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:31.232003 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:31.231489 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27pfj" podUID="3afa8f46-c8f3-4b0b-a33e-42d05fefc58f" Jul 02 19:03:31.232003 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:31.231382 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:31.232003 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:31.231546 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6zb2" podUID="7c305fe4-c346-4c57-b6c6-1dea6f663801" Jul 02 19:03:31.232003 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:31.231625 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8sprk" podUID="9332d088-8ecc-4c56-a0d8-6d35d5acda58" Jul 02 19:03:32.072418 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.072391 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-101.ec2.internal" event="NodeReady" Jul 02 19:03:32.072581 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.072507 2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 19:03:32.107373 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.107338 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c7bdd876f-4rccg"] Jul 02 19:03:32.125477 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.125447 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mckjl"] Jul 02 19:03:32.125636 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.125581 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.128447 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.128424 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 19:03:32.128605 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.128498 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 19:03:32.128605 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.128526 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 19:03:32.128605 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.128575 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-k9mwx\"" Jul 02 19:03:32.138105 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.136386 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 19:03:32.149518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.149493 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c7bdd876f-4rccg"] Jul 02 19:03:32.149518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.149525 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pqz7k"] Jul 02 19:03:32.149676 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.149611 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mckjl" Jul 02 19:03:32.151940 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.151917 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 19:03:32.152062 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.151999 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 19:03:32.152062 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.152018 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 19:03:32.152174 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.152139 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cf2kh\"" Jul 02 19:03:32.166955 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.166923 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1da140cc-562a-4c88-b899-3001afc4e989-ca-trust-extracted\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.166955 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.166956 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1da140cc-562a-4c88-b899-3001afc4e989-bound-sa-token\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.167187 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.166988 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkl7r\" (UniqueName: \"kubernetes.io/projected/aa480a02-4284-406b-b0a4-b72a2fc9800d-kube-api-access-hkl7r\") pod \"ingress-canary-mckjl\" (UID: \"aa480a02-4284-406b-b0a4-b72a2fc9800d\") " pod="openshift-ingress-canary/ingress-canary-mckjl" Jul 02 19:03:32.167187 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.167024 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1da140cc-562a-4c88-b899-3001afc4e989-trusted-ca\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.167187 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.167043 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1da140cc-562a-4c88-b899-3001afc4e989-installation-pull-secrets\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.167187 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.167060 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1da140cc-562a-4c88-b899-3001afc4e989-registry-certificates\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.167187 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.167138 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1da140cc-562a-4c88-b899-3001afc4e989-registry-tls\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.167443 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.167194 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1da140cc-562a-4c88-b899-3001afc4e989-image-registry-private-configuration\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.167443 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.167225 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbn95\" (UniqueName: \"kubernetes.io/projected/1da140cc-562a-4c88-b899-3001afc4e989-kube-api-access-gbn95\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.167443 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.167253 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa480a02-4284-406b-b0a4-b72a2fc9800d-cert\") pod \"ingress-canary-mckjl\" (UID: \"aa480a02-4284-406b-b0a4-b72a2fc9800d\") " pod="openshift-ingress-canary/ingress-canary-mckjl" Jul 02 19:03:32.169486 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.169469 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mckjl"] Jul 02 19:03:32.169535 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.169503 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pqz7k"] Jul 02 19:03:32.169588 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.169574 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.171952 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.171923 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 19:03:32.172082 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.172040 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-psrnn\"" Jul 02 19:03:32.172164 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.172148 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 19:03:32.172223 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.172211 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 19:03:32.172302 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.172287 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 19:03:32.219897 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.219869 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c2szs"] Jul 02 19:03:32.237828 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.237802 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c2szs"] Jul 02 19:03:32.238268 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.237919 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.240282 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.240267 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 19:03:32.240345 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.240269 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 19:03:32.240532 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.240519 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g97vz\"" Jul 02 19:03:32.268373 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268341 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbn95\" (UniqueName: \"kubernetes.io/projected/1da140cc-562a-4c88-b899-3001afc4e989-kube-api-access-gbn95\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.268373 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268373 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa480a02-4284-406b-b0a4-b72a2fc9800d-cert\") pod \"ingress-canary-mckjl\" (UID: \"aa480a02-4284-406b-b0a4-b72a2fc9800d\") " pod="openshift-ingress-canary/ingress-canary-mckjl" Jul 02 19:03:32.268617 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268393 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-data-volume\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.268617 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268410 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-crio-socket\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.268617 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268457 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9309e0d2-176b-4f90-bb3c-fc117cdea733-tmp-dir\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.268617 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268583 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1da140cc-562a-4c88-b899-3001afc4e989-ca-trust-extracted\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.268617 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268618 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1da140cc-562a-4c88-b899-3001afc4e989-bound-sa-token\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.268860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268648 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.268860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268693 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phmx6\" (UniqueName: \"kubernetes.io/projected/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-kube-api-access-phmx6\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.268860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268721 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkl7r\" (UniqueName: \"kubernetes.io/projected/aa480a02-4284-406b-b0a4-b72a2fc9800d-kube-api-access-hkl7r\") pod \"ingress-canary-mckjl\" (UID: \"aa480a02-4284-406b-b0a4-b72a2fc9800d\") " pod="openshift-ingress-canary/ingress-canary-mckjl" Jul 02 19:03:32.268860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268806 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1da140cc-562a-4c88-b899-3001afc4e989-trusted-ca\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.268860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268847 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1da140cc-562a-4c88-b899-3001afc4e989-installation-pull-secrets\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.269119 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268871 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9309e0d2-176b-4f90-bb3c-fc117cdea733-metrics-tls\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.269119 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.268897 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.269119 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.269004 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1da140cc-562a-4c88-b899-3001afc4e989-registry-certificates\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.269119 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.269055 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1da140cc-562a-4c88-b899-3001afc4e989-ca-trust-extracted\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.269119 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.269090 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1da140cc-562a-4c88-b899-3001afc4e989-registry-tls\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.269363 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.269148 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9309e0d2-176b-4f90-bb3c-fc117cdea733-config-volume\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.269363 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.269176 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt6gr\" (UniqueName: \"kubernetes.io/projected/9309e0d2-176b-4f90-bb3c-fc117cdea733-kube-api-access-vt6gr\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.269363 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.269216 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1da140cc-562a-4c88-b899-3001afc4e989-image-registry-private-configuration\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.269831 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.269805 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1da140cc-562a-4c88-b899-3001afc4e989-registry-certificates\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.269995 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.269971 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1da140cc-562a-4c88-b899-3001afc4e989-trusted-ca\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.272936 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.272911 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1da140cc-562a-4c88-b899-3001afc4e989-installation-pull-secrets\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.272936 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.272925 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1da140cc-562a-4c88-b899-3001afc4e989-image-registry-private-configuration\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.272936 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.272933 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1da140cc-562a-4c88-b899-3001afc4e989-registry-tls\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.273122 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.273008 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa480a02-4284-406b-b0a4-b72a2fc9800d-cert\") pod \"ingress-canary-mckjl\" (UID: \"aa480a02-4284-406b-b0a4-b72a2fc9800d\") " pod="openshift-ingress-canary/ingress-canary-mckjl" Jul 02 19:03:32.276278 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.276253 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1da140cc-562a-4c88-b899-3001afc4e989-bound-sa-token\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.276383 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.276363 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkl7r\" (UniqueName: \"kubernetes.io/projected/aa480a02-4284-406b-b0a4-b72a2fc9800d-kube-api-access-hkl7r\") pod \"ingress-canary-mckjl\" (UID: \"aa480a02-4284-406b-b0a4-b72a2fc9800d\") " pod="openshift-ingress-canary/ingress-canary-mckjl" Jul 02 19:03:32.276453 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.276440 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbn95\" (UniqueName: \"kubernetes.io/projected/1da140cc-562a-4c88-b899-3001afc4e989-kube-api-access-gbn95\") pod \"image-registry-7c7bdd876f-4rccg\" (UID: \"1da140cc-562a-4c88-b899-3001afc4e989\") " pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.369956 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.369884 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.369956 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.369929 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phmx6\" (UniqueName: \"kubernetes.io/projected/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-kube-api-access-phmx6\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.370199 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370061 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9309e0d2-176b-4f90-bb3c-fc117cdea733-metrics-tls\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.370272 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370219 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.370272 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370259 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9309e0d2-176b-4f90-bb3c-fc117cdea733-config-volume\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.370372 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370274 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6gr\" (UniqueName: \"kubernetes.io/projected/9309e0d2-176b-4f90-bb3c-fc117cdea733-kube-api-access-vt6gr\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.370372 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370307 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-data-volume\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.370372 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370331 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-crio-socket\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.370372 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370356 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9309e0d2-176b-4f90-bb3c-fc117cdea733-tmp-dir\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.370799 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370554 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-crio-socket\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.370799 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370635 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-data-volume\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.370908 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370860 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9309e0d2-176b-4f90-bb3c-fc117cdea733-tmp-dir\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.370908 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370881 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.370908 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.370899 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9309e0d2-176b-4f90-bb3c-fc117cdea733-config-volume\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.372189 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.372165 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.372292 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.372277 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9309e0d2-176b-4f90-bb3c-fc117cdea733-metrics-tls\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.378635 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.378615 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt6gr\" (UniqueName: \"kubernetes.io/projected/9309e0d2-176b-4f90-bb3c-fc117cdea733-kube-api-access-vt6gr\") pod \"dns-default-c2szs\" (UID: \"9309e0d2-176b-4f90-bb3c-fc117cdea733\") " pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.378734 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.378666 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phmx6\" (UniqueName: \"kubernetes.io/projected/5f92e3f7-20e6-440b-ad6a-b5fb175b19a3-kube-api-access-phmx6\") pod \"insights-runtime-extractor-pqz7k\" (UID: \"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3\") " pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.439290 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.439254 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:32.457452 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.457416 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mckjl" Jul 02 19:03:32.477237 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.477216 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pqz7k" Jul 02 19:03:32.547230 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.546789 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:32.668201 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.667970 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mckjl"] Jul 02 19:03:32.671023 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:32.670983 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa480a02_4284_406b_b0a4_b72a2fc9800d.slice/crio-8d6c5c7e86f664bac672992bdf217260866155ffbc7701c4b23f0db5f6cad50a WatchSource:0}: Error finding container 8d6c5c7e86f664bac672992bdf217260866155ffbc7701c4b23f0db5f6cad50a: Status 404 returned error can't find the container with id 8d6c5c7e86f664bac672992bdf217260866155ffbc7701c4b23f0db5f6cad50a Jul 02 19:03:32.683954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.683645 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c7bdd876f-4rccg"] Jul 02 19:03:32.687178 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:32.687151 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da140cc_562a_4c88_b899_3001afc4e989.slice/crio-d7965cd7c68f0daa6efc96a82f35fff70a16babd0e1e1a6e1fd64f4c2e76e4be WatchSource:0}: Error finding container d7965cd7c68f0daa6efc96a82f35fff70a16babd0e1e1a6e1fd64f4c2e76e4be: Status 404 returned error can't find the container with id d7965cd7c68f0daa6efc96a82f35fff70a16babd0e1e1a6e1fd64f4c2e76e4be Jul 02 19:03:32.715337 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.715311 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pqz7k"] Jul 02 19:03:32.715745 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:32.715724 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f92e3f7_20e6_440b_ad6a_b5fb175b19a3.slice/crio-ee48af37b3f064a941bc6f3e31ae923ed7a4f9f80399172fcf6f87d284648b7d WatchSource:0}: Error finding container ee48af37b3f064a941bc6f3e31ae923ed7a4f9f80399172fcf6f87d284648b7d: Status 404 returned error can't find the container with id ee48af37b3f064a941bc6f3e31ae923ed7a4f9f80399172fcf6f87d284648b7d Jul 02 19:03:32.733387 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:32.733360 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c2szs"] Jul 02 19:03:32.739977 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:32.739954 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9309e0d2_176b_4f90_bb3c_fc117cdea733.slice/crio-8edf55bd7f8911f36b02709842ab60e505200b7a8a8f0275d3258721ba756516 WatchSource:0}: Error finding container 8edf55bd7f8911f36b02709842ab60e505200b7a8a8f0275d3258721ba756516: Status 404 returned error can't find the container with id 8edf55bd7f8911f36b02709842ab60e505200b7a8a8f0275d3258721ba756516 Jul 02 19:03:33.236342 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.236306 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:33.236937 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.236814 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:33.237089 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.236968 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:33.239789 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.239771 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 19:03:33.240685 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.240663 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 19:03:33.240787 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.240696 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xfkkw\"" Jul 02 19:03:33.248662 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.248478 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 19:03:33.248662 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.248561 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 19:03:33.248997 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.248807 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bswjt\"" Jul 02 19:03:33.408555 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.408514 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" event={"ID":"1da140cc-562a-4c88-b899-3001afc4e989","Type":"ContainerStarted","Data":"28ea517e5423bfa662313a76765f537b7f05cf370181c81ffd6db62383beeaec"} Jul 02 19:03:33.408727 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.408580 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" event={"ID":"1da140cc-562a-4c88-b899-3001afc4e989","Type":"ContainerStarted","Data":"d7965cd7c68f0daa6efc96a82f35fff70a16babd0e1e1a6e1fd64f4c2e76e4be"} Jul 02 19:03:33.409299 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.409280 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:33.411389 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.411359 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pqz7k" event={"ID":"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3","Type":"ContainerStarted","Data":"ee48af37b3f064a941bc6f3e31ae923ed7a4f9f80399172fcf6f87d284648b7d"} Jul 02 19:03:33.413903 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.413874 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mckjl" event={"ID":"aa480a02-4284-406b-b0a4-b72a2fc9800d","Type":"ContainerStarted","Data":"8d6c5c7e86f664bac672992bdf217260866155ffbc7701c4b23f0db5f6cad50a"} Jul 02 19:03:33.417240 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.417214 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c2szs" event={"ID":"9309e0d2-176b-4f90-bb3c-fc117cdea733","Type":"ContainerStarted","Data":"8edf55bd7f8911f36b02709842ab60e505200b7a8a8f0275d3258721ba756516"} Jul 02 19:03:33.431023 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:33.430966 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" podStartSLOduration=7.4309452799999995 podStartE2EDuration="7.43094528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:03:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:33.430132129 +0000 UTC m=+50.721469026" watchObservedRunningTime="2026-07-02 19:03:33.43094528 +0000 UTC m=+50.722282178" Jul 02 19:03:38.431781 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.431742 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pqz7k" event={"ID":"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3","Type":"ContainerStarted","Data":"e1a76be5e2587f38fc0b57b371b78e6df005d30114822900a23b9ba306d4d2e8"} Jul 02 19:03:38.433778 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.433557 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mckjl" event={"ID":"aa480a02-4284-406b-b0a4-b72a2fc9800d","Type":"ContainerStarted","Data":"bea4d8229d1fef5c15a0256d6f63ac94d01096bcaedf0d55b4a015b075ee71c2"} Jul 02 19:03:38.435314 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.435284 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c2szs" event={"ID":"9309e0d2-176b-4f90-bb3c-fc117cdea733","Type":"ContainerStarted","Data":"7561d661b384f0fc61d10fca7489e8ece3b438eb6c9ed953a2662c996148c461"} Jul 02 19:03:38.435438 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.435319 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c2szs" event={"ID":"9309e0d2-176b-4f90-bb3c-fc117cdea733","Type":"ContainerStarted","Data":"836a3506c520a0b16f77fbc3e11b2b908a2e8bf23900ea29a6526b24aba7a738"} Jul 02 19:03:38.435836 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.435677 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:38.455765 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.455706 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mckjl" podStartSLOduration=0.894297095 podStartE2EDuration="6.455689802s" totalImagesPullingTime="5.561392707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:03:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:38.455339872 +0000 UTC m=+55.746676768" watchObservedRunningTime="2026-07-02 19:03:38.455689802 +0000 UTC m=+55.747026701" Jul 02 19:03:38.485226 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.485099 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c2szs" podStartSLOduration=1.011620245 podStartE2EDuration="6.485058683s" totalImagesPullingTime="5.473438438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:03:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:38.484104823 +0000 UTC m=+55.775441724" watchObservedRunningTime="2026-07-02 19:03:38.485058683 +0000 UTC m=+55.776395585" Jul 02 19:03:38.609036 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.609000 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg"] Jul 02 19:03:38.610790 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.610768 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg" Jul 02 19:03:38.615254 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.615237 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 02 19:03:38.617544 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.617527 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-fl5m9\"" Jul 02 19:03:38.627276 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.627253 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg"] Jul 02 19:03:38.725084 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.725027 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/34484f96-7f73-406e-af29-d1e7621e4580-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h7hwg\" (UID: \"34484f96-7f73-406e-af29-d1e7621e4580\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg" Jul 02 19:03:38.826294 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.826252 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/34484f96-7f73-406e-af29-d1e7621e4580-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h7hwg\" (UID: \"34484f96-7f73-406e-af29-d1e7621e4580\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg" Jul 02 19:03:38.830307 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.830283 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/34484f96-7f73-406e-af29-d1e7621e4580-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-h7hwg\" (UID: \"34484f96-7f73-406e-af29-d1e7621e4580\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg" Jul 02 19:03:38.919277 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:38.919217 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg" Jul 02 19:03:39.046754 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:39.046726 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg"] Jul 02 19:03:39.049124 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:39.049101 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34484f96_7f73_406e_af29_d1e7621e4580.slice/crio-5f89dc7c85fa9a6b7e1c2d44eb6a2acd6ca091c4c80f7e67e4feef2c1b51a19c WatchSource:0}: Error finding container 5f89dc7c85fa9a6b7e1c2d44eb6a2acd6ca091c4c80f7e67e4feef2c1b51a19c: Status 404 returned error can't find the container with id 5f89dc7c85fa9a6b7e1c2d44eb6a2acd6ca091c4c80f7e67e4feef2c1b51a19c Jul 02 19:03:39.441445 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:39.441404 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg" event={"ID":"34484f96-7f73-406e-af29-d1e7621e4580","Type":"ContainerStarted","Data":"5f89dc7c85fa9a6b7e1c2d44eb6a2acd6ca091c4c80f7e67e4feef2c1b51a19c"} Jul 02 19:03:39.443498 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:39.443461 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pqz7k" event={"ID":"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3","Type":"ContainerStarted","Data":"4fa8b37cca3120d76ba2fe365d399a4a8e3684836ae7ca95ff79d8cec4eb44b3"} Jul 02 19:03:41.455341 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:41.455302 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg" event={"ID":"34484f96-7f73-406e-af29-d1e7621e4580","Type":"ContainerStarted","Data":"42f13faa3bc0b27245bd6b5865bf4e0677cf28fd371c944f35c98efa402627f6"} Jul 02 19:03:41.455809 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:41.455497 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg" Jul 02 19:03:41.457547 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:41.457519 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pqz7k" event={"ID":"5f92e3f7-20e6-440b-ad6a-b5fb175b19a3","Type":"ContainerStarted","Data":"a20851674a320c54c54b237085eaac970529c66d60404cb7ad02bdca5df63488"} Jul 02 19:03:41.460766 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:41.460751 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg" Jul 02 19:03:41.474217 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:41.474170 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-h7hwg" podStartSLOduration=1.2624704 podStartE2EDuration="3.474156478s" totalImagesPullingTime="2.211686078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:41.473673243 +0000 UTC m=+58.765010144" watchObservedRunningTime="2026-07-02 19:03:41.474156478 +0000 UTC m=+58.765493355" Jul 02 19:03:41.519859 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:41.519797 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pqz7k" podStartSLOduration=1.090924277 podStartE2EDuration="9.519782616s" totalImagesPullingTime="8.428858339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:41.518902547 +0000 UTC m=+58.810239660" watchObservedRunningTime="2026-07-02 19:03:41.519782616 +0000 UTC m=+58.811119516" Jul 02 19:03:44.366220 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:44.366184 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kr7sw" Jul 02 19:03:48.113448 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.113414 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9gmrb"] Jul 02 19:03:48.116585 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.116563 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm"] Jul 02 19:03:48.116706 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.116688 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.118731 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.118710 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.119479 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.119461 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 19:03:48.119888 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.119867 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 19:03:48.120403 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.120372 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 19:03:48.120635 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.120419 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 19:03:48.120635 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.120468 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2g2r7\"" Jul 02 19:03:48.120635 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.120589 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 19:03:48.120839 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.120770 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 19:03:48.122168 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.122144 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 02 19:03:48.122278 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.122195 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 02 19:03:48.122447 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.122431 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qqxlg\"" Jul 02 19:03:48.122544 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.122528 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 02 19:03:48.161451 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.161421 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm"] Jul 02 19:03:48.199240 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199206 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bx8g\" (UniqueName: \"kubernetes.io/projected/96f57550-6488-4662-ad6c-0241cd5035f1-kube-api-access-5bx8g\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.199416 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199268 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.199416 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199296 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-tls\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.199416 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199321 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-sys\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.199416 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199341 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.199416 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199381 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.199416 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199406 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-root\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.199718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199424 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-wtmp\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.199718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199457 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m9n4\" (UniqueName: \"kubernetes.io/projected/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-kube-api-access-8m9n4\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.199718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199572 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/96f57550-6488-4662-ad6c-0241cd5035f1-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.199718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199623 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96f57550-6488-4662-ad6c-0241cd5035f1-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.199718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199651 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-textfile\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.199718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199680 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.199718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199711 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-metrics-client-ca\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.199950 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.199739 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.300461 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300431 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96f57550-6488-4662-ad6c-0241cd5035f1-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.300461 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300463 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-textfile\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.300651 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300482 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.300651 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300503 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-metrics-client-ca\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.300651 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300522 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.300651 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300642 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bx8g\" (UniqueName: \"kubernetes.io/projected/96f57550-6488-4662-ad6c-0241cd5035f1-kube-api-access-5bx8g\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.300832 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300704 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.300832 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300737 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-tls\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.300832 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300784 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-sys\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.300832 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300808 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.301021 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300841 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.301021 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300861 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-root\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.301021 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300879 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-textfile\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.301021 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.301005 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-wtmp\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.301312 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.300881 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-wtmp\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.301312 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.301057 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m9n4\" (UniqueName: \"kubernetes.io/projected/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-kube-api-access-8m9n4\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.301312 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.301156 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/96f57550-6488-4662-ad6c-0241cd5035f1-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.301312 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.301253 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-metrics-client-ca\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.301312 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.301287 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.301542 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.301349 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-root\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.301542 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:48.301358 2571 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 02 19:03:48.301542 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.301390 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96f57550-6488-4662-ad6c-0241cd5035f1-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.301542 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:48.301414 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-tls podName:96f57550-6488-4662-ad6c-0241cd5035f1 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:48.801395793 +0000 UTC m=+66.092732686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-g9lwm" (UID: "96f57550-6488-4662-ad6c-0241cd5035f1") : secret "kube-state-metrics-tls" not found Jul 02 19:03:48.301542 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.301446 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.301542 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.301462 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-sys\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.301542 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.301488 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/96f57550-6488-4662-ad6c-0241cd5035f1-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.303148 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.303126 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.303246 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.303175 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.303290 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.303261 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-node-exporter-tls\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.310268 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.310239 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m9n4\" (UniqueName: \"kubernetes.io/projected/a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4-kube-api-access-8m9n4\") pod \"node-exporter-9gmrb\" (UID: \"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4\") " pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.310487 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.310469 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bx8g\" (UniqueName: \"kubernetes.io/projected/96f57550-6488-4662-ad6c-0241cd5035f1-kube-api-access-5bx8g\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.427257 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.427184 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9gmrb" Jul 02 19:03:48.438422 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:48.438395 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89c3e35_f2b0_4d44_8e34_7b23f6cb05f4.slice/crio-83af0d7f365493e0fdb548465be5a6b551e4f47f28c4c48ef6f204fc4748c8bb WatchSource:0}: Error finding container 83af0d7f365493e0fdb548465be5a6b551e4f47f28c4c48ef6f204fc4748c8bb: Status 404 returned error can't find the container with id 83af0d7f365493e0fdb548465be5a6b551e4f47f28c4c48ef6f204fc4748c8bb Jul 02 19:03:48.479535 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.479503 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9gmrb" event={"ID":"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4","Type":"ContainerStarted","Data":"83af0d7f365493e0fdb548465be5a6b551e4f47f28c4c48ef6f204fc4748c8bb"} Jul 02 19:03:48.804741 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.804697 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.806992 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.806972 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f57550-6488-4662-ad6c-0241cd5035f1-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-g9lwm\" (UID: \"96f57550-6488-4662-ad6c-0241cd5035f1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:48.905214 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.905171 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:48.907457 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.907439 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 19:03:48.918243 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:48.918216 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3afa8f46-c8f3-4b0b-a33e-42d05fefc58f-metrics-certs\") pod \"network-metrics-daemon-27pfj\" (UID: \"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f\") " pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:49.006642 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.006601 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:49.008833 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.008809 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 19:03:49.020283 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.020229 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9332d088-8ecc-4c56-a0d8-6d35d5acda58-original-pull-secret\") pod \"global-pull-secret-syncer-8sprk\" (UID: \"9332d088-8ecc-4c56-a0d8-6d35d5acda58\") " pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:49.032289 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.032263 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" Jul 02 19:03:49.107624 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.107585 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrlj\" (UniqueName: \"kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj\") pod \"network-check-target-z6zb2\" (UID: \"7c305fe4-c346-4c57-b6c6-1dea6f663801\") " pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:49.110628 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.110389 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 19:03:49.120790 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.120740 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 19:03:49.131673 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.131627 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbrlj\" (UniqueName: \"kubernetes.io/projected/7c305fe4-c346-4c57-b6c6-1dea6f663801-kube-api-access-wbrlj\") pod \"network-check-target-z6zb2\" (UID: \"7c305fe4-c346-4c57-b6c6-1dea6f663801\") " pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:49.158618 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.158593 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xfkkw\"" Jul 02 19:03:49.162965 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.162942 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:03:49.164985 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.164964 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8sprk" Jul 02 19:03:49.167102 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.167066 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27pfj" Jul 02 19:03:49.168428 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.168415 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.170787 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.170759 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 19:03:49.170787 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.170777 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 19:03:49.171018 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.170827 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 19:03:49.171018 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.170832 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 19:03:49.171018 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.170966 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 19:03:49.171288 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.171182 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 19:03:49.171288 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.171187 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 19:03:49.171288 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.171238 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 19:03:49.171288 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.171285 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 19:03:49.171477 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.171297 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-65l4w\"" Jul 02 19:03:49.173762 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.173746 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bswjt\"" Jul 02 19:03:49.180514 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.180385 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm"] Jul 02 19:03:49.187921 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.181325 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:49.187921 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.186918 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:03:49.309921 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.309694 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.309921 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.309736 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.309921 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.309760 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.309921 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.309777 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-config-out\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.309921 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.309812 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-config-volume\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.309921 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.309836 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.309921 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.309858 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.309921 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.309881 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88lg7\" (UniqueName: \"kubernetes.io/projected/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-kube-api-access-88lg7\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.310431 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.309952 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.310431 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.310045 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-web-config\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.310431 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.310119 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.310431 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.310149 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-tls-assets\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.310431 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.310195 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.330601 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.330572 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8sprk"] Jul 02 19:03:49.334956 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:49.334212 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9332d088_8ecc_4c56_a0d8_6d35d5acda58.slice/crio-dda4dcdea7cd33f85547db8e46f2f48a9da4356a2db728e54e9af7c7884a2340 WatchSource:0}: Error finding container dda4dcdea7cd33f85547db8e46f2f48a9da4356a2db728e54e9af7c7884a2340: Status 404 returned error can't find the container with id dda4dcdea7cd33f85547db8e46f2f48a9da4356a2db728e54e9af7c7884a2340 Jul 02 19:03:49.351934 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.350948 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-27pfj"] Jul 02 19:03:49.358216 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:49.358183 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3afa8f46_c8f3_4b0b_a33e_42d05fefc58f.slice/crio-c8b9fe0ce2fd13ee9bf9d720eeb60b1a7156162c7a263c03f1c84157ddf61e4a WatchSource:0}: Error finding container c8b9fe0ce2fd13ee9bf9d720eeb60b1a7156162c7a263c03f1c84157ddf61e4a: Status 404 returned error can't find the container with id c8b9fe0ce2fd13ee9bf9d720eeb60b1a7156162c7a263c03f1c84157ddf61e4a Jul 02 19:03:49.373085 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.373040 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z6zb2"] Jul 02 19:03:49.376176 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:49.376147 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c305fe4_c346_4c57_b6c6_1dea6f663801.slice/crio-53dd237728b6c5b4d9ca1e7ff0218760cf987896a73db31ad09c140c580bb4f1 WatchSource:0}: Error finding container 53dd237728b6c5b4d9ca1e7ff0218760cf987896a73db31ad09c140c580bb4f1: Status 404 returned error can't find the container with id 53dd237728b6c5b4d9ca1e7ff0218760cf987896a73db31ad09c140c580bb4f1 Jul 02 19:03:49.411379 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.411341 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-tls-assets\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.411585 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.411390 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.411585 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.411419 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.411585 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:49.411535 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-alertmanager-trusted-ca-bundle podName:241b3746-c7e5-43d4-9e38-cc0b2d7a9b50 nodeName:}" failed. No retries permitted until 2026-07-02 19:03:49.911517166 +0000 UTC m=+67.202854044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "241b3746-c7e5-43d4-9e38-cc0b2d7a9b50") : configmap references non-existent config key: ca-bundle.crt Jul 02 19:03:49.411757 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.411597 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.411757 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.411637 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.411757 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.411658 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-config-out\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.411757 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.411725 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-config-volume\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.411757 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.411753 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.412003 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.411777 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.413244 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.412371 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88lg7\" (UniqueName: \"kubernetes.io/projected/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-kube-api-access-88lg7\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.413244 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.412415 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.413244 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.412464 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.413244 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.412514 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-web-config\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.413244 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.412561 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.413244 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.412871 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.414587 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.414556 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-config-out\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.415389 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.415003 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-config-volume\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.415389 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.415108 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.415389 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.415143 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.415565 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.415411 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.415565 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.415491 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-tls-assets\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.415819 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.415795 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.415819 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.415815 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.416366 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.416335 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-web-config\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.421218 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.421195 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88lg7\" (UniqueName: \"kubernetes.io/projected/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-kube-api-access-88lg7\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.452793 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.452759 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c2szs" Jul 02 19:03:49.483779 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.483738 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" event={"ID":"96f57550-6488-4662-ad6c-0241cd5035f1","Type":"ContainerStarted","Data":"83a82d75c91ef04de7e9a56c4a0b75170c8fb09e83efc386d46145e6a1b6e0f0"} Jul 02 19:03:49.485145 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.485085 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z6zb2" event={"ID":"7c305fe4-c346-4c57-b6c6-1dea6f663801","Type":"ContainerStarted","Data":"53dd237728b6c5b4d9ca1e7ff0218760cf987896a73db31ad09c140c580bb4f1"} Jul 02 19:03:49.486699 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.486640 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-27pfj" event={"ID":"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f","Type":"ContainerStarted","Data":"c8b9fe0ce2fd13ee9bf9d720eeb60b1a7156162c7a263c03f1c84157ddf61e4a"} Jul 02 19:03:49.488105 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.488055 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8sprk" event={"ID":"9332d088-8ecc-4c56-a0d8-6d35d5acda58","Type":"ContainerStarted","Data":"dda4dcdea7cd33f85547db8e46f2f48a9da4356a2db728e54e9af7c7884a2340"} Jul 02 19:03:49.916667 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.916625 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:49.917443 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:49.917425 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241b3746-c7e5-43d4-9e38-cc0b2d7a9b50-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:50.080529 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:50.080509 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:03:50.255806 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:50.255776 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:03:50.258606 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:50.258576 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241b3746_c7e5_43d4_9e38_cc0b2d7a9b50.slice/crio-3edb241f35d614ec4a1ec0e31d76bbb38d626d880717cfb7b9923d1550b3d57a WatchSource:0}: Error finding container 3edb241f35d614ec4a1ec0e31d76bbb38d626d880717cfb7b9923d1550b3d57a: Status 404 returned error can't find the container with id 3edb241f35d614ec4a1ec0e31d76bbb38d626d880717cfb7b9923d1550b3d57a Jul 02 19:03:50.496733 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:50.496609 2571 generic.go:356] "Generic (PLEG): container finished" podID="a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4" containerID="0f1a3428e5be0de6b7ade9c96530f856200c96edad2a63ab633efd020b968620" exitCode=0 Jul 02 19:03:50.496733 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:50.496721 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9gmrb" event={"ID":"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4","Type":"ContainerDied","Data":"0f1a3428e5be0de6b7ade9c96530f856200c96edad2a63ab633efd020b968620"} Jul 02 19:03:50.501686 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:50.501655 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50","Type":"ContainerStarted","Data":"3edb241f35d614ec4a1ec0e31d76bbb38d626d880717cfb7b9923d1550b3d57a"} Jul 02 19:03:51.509339 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:51.509297 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9gmrb" event={"ID":"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4","Type":"ContainerStarted","Data":"97b26f76bbafda6699da24d19a11f560082162ec177e03e124169586b854bb6f"} Jul 02 19:03:51.509787 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:51.509368 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9gmrb" event={"ID":"a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4","Type":"ContainerStarted","Data":"8003a07c9820ab41d8b3983ad9a111d03082b144fbd089448b53ad2c7a23cca8"} Jul 02 19:03:51.534255 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:51.534183 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9gmrb" podStartSLOduration=1.773891844 podStartE2EDuration="3.534160886s" totalImagesPullingTime="1.633190889s" totalInitContainerRuntime="127.078153ms" isStatefulPod=true podCreationTimestamp="2026-07-02 19:03:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:51.532102982 +0000 UTC m=+68.823439892" watchObservedRunningTime="2026-07-02 19:03:51.534160886 +0000 UTC m=+68.825497786" Jul 02 19:03:52.445727 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.445629 2571 patch_prober.go:28] interesting pod/image-registry-7c7bdd876f-4rccg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 19:03:52.445727 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.445696 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" podUID="1da140cc-562a-4c88-b899-3001afc4e989" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 19:03:52.491545 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.491508 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5bd5947889-lfh8v"] Jul 02 19:03:52.494015 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.493992 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.497523 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.497499 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 02 19:03:52.497523 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.497499 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 19:03:52.497701 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.497528 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 02 19:03:52.497701 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.497661 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-emd1vjobj73n2\"" Jul 02 19:03:52.497860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.497829 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 02 19:03:52.498370 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.498051 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-cxb9c\"" Jul 02 19:03:52.504653 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.504453 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bd5947889-lfh8v"] Jul 02 19:03:52.522491 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.522365 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" event={"ID":"96f57550-6488-4662-ad6c-0241cd5035f1","Type":"ContainerStarted","Data":"e727ab53afdc332332dc3258fb9cbd8bb679dd8139aad766eb59588888edf078"} Jul 02 19:03:52.522491 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.522430 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" event={"ID":"96f57550-6488-4662-ad6c-0241cd5035f1","Type":"ContainerStarted","Data":"987c20d303364656ea09f7ff770f5b700824c6b15f2dcffb980831800d196384"} Jul 02 19:03:52.522491 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.522460 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" event={"ID":"96f57550-6488-4662-ad6c-0241cd5035f1","Type":"ContainerStarted","Data":"fcee3f4a55e5a83c422624ca9eaca0d3bb2c9b251d878d51ce3cfcc8518daeb4"} Jul 02 19:03:52.524581 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.524553 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-27pfj" event={"ID":"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f","Type":"ContainerStarted","Data":"0883c9e358e04883f40f9759e5b0fbf33def8d5e6931bc858dbc6969c954e6c9"} Jul 02 19:03:52.524706 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.524608 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-27pfj" event={"ID":"3afa8f46-c8f3-4b0b-a33e-42d05fefc58f","Type":"ContainerStarted","Data":"6fded9dac7bebfabfd93cd40c8194373b2a67cc87fe9d79bfa57fb8188b9bf40"} Jul 02 19:03:52.526696 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.526667 2571 generic.go:356] "Generic (PLEG): container finished" podID="241b3746-c7e5-43d4-9e38-cc0b2d7a9b50" containerID="16f2ebe97507dd4a082ceb5a1d54e82fb5adc8a0ac651bb9da63e902feabe674" exitCode=0 Jul 02 19:03:52.526805 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.526788 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50","Type":"ContainerDied","Data":"16f2ebe97507dd4a082ceb5a1d54e82fb5adc8a0ac651bb9da63e902feabe674"} Jul 02 19:03:52.542516 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.542458 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9lwm" podStartSLOduration=1.925537132 podStartE2EDuration="4.542439124s" totalImagesPullingTime="2.616901992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:03:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:52.542201476 +0000 UTC m=+69.833538377" watchObservedRunningTime="2026-07-02 19:03:52.542439124 +0000 UTC m=+69.833776028" Jul 02 19:03:52.560860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.560344 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-27pfj" podStartSLOduration=67.117600583 podStartE2EDuration="1m9.560322369s" totalImagesPullingTime="2.442721786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:52.559006835 +0000 UTC m=+69.850343739" watchObservedRunningTime="2026-07-02 19:03:52.560322369 +0000 UTC m=+69.851659270" Jul 02 19:03:52.644924 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.643687 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.644924 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.643757 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt6lf\" (UniqueName: \"kubernetes.io/projected/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-kube-api-access-lt6lf\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.644924 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.643785 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-client-ca-bundle\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.644924 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.643946 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-audit-log\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.644924 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.644029 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-metrics-server-audit-profiles\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.644924 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.644173 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-secret-metrics-server-client-certs\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.644924 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.644275 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-secret-metrics-server-tls\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.745554 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.745461 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt6lf\" (UniqueName: \"kubernetes.io/projected/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-kube-api-access-lt6lf\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.745554 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.745511 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-client-ca-bundle\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.745776 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.745573 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-audit-log\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.745776 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.745607 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-metrics-server-audit-profiles\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.745776 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.745643 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-secret-metrics-server-client-certs\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.745776 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.745677 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-secret-metrics-server-tls\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.745776 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.745728 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.746115 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.746096 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-audit-log\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.746441 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.746414 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.746772 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.746750 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-metrics-server-audit-profiles\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.748542 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.748516 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-secret-metrics-server-tls\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.748639 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.748585 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-client-ca-bundle\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.748738 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.748714 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-secret-metrics-server-client-certs\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.756293 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.756261 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt6lf\" (UniqueName: \"kubernetes.io/projected/1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3-kube-api-access-lt6lf\") pod \"metrics-server-5bd5947889-lfh8v\" (UID: \"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3\") " pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.809844 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.809806 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:03:52.849101 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.849057 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj"] Jul 02 19:03:52.852038 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.852016 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" Jul 02 19:03:52.854749 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.854687 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 02 19:03:52.855039 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.855019 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6pnkv\"" Jul 02 19:03:52.863109 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.862558 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj"] Jul 02 19:03:52.946954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.946882 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfstq\" (UniqueName: \"kubernetes.io/projected/a75a2281-4146-464f-bdda-8f221a98f52d-kube-api-access-kfstq\") pod \"monitoring-plugin-d5dcff7cf-bp5hj\" (UID: \"a75a2281-4146-464f-bdda-8f221a98f52d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" Jul 02 19:03:52.946954 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.946926 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a75a2281-4146-464f-bdda-8f221a98f52d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bp5hj\" (UID: \"a75a2281-4146-464f-bdda-8f221a98f52d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" Jul 02 19:03:52.969924 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:52.969886 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bd5947889-lfh8v"] Jul 02 19:03:53.047558 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:53.047524 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfstq\" (UniqueName: \"kubernetes.io/projected/a75a2281-4146-464f-bdda-8f221a98f52d-kube-api-access-kfstq\") pod \"monitoring-plugin-d5dcff7cf-bp5hj\" (UID: \"a75a2281-4146-464f-bdda-8f221a98f52d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" Jul 02 19:03:53.047757 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:53.047569 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a75a2281-4146-464f-bdda-8f221a98f52d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bp5hj\" (UID: \"a75a2281-4146-464f-bdda-8f221a98f52d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" Jul 02 19:03:53.047808 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:53.047767 2571 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 02 19:03:53.047851 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:03:53.047842 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a75a2281-4146-464f-bdda-8f221a98f52d-monitoring-plugin-cert podName:a75a2281-4146-464f-bdda-8f221a98f52d nodeName:}" failed. No retries permitted until 2026-07-02 19:03:53.547822855 +0000 UTC m=+70.839159752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/a75a2281-4146-464f-bdda-8f221a98f52d-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-bp5hj" (UID: "a75a2281-4146-464f-bdda-8f221a98f52d") : secret "monitoring-plugin-cert" not found Jul 02 19:03:53.056414 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:53.056387 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfstq\" (UniqueName: \"kubernetes.io/projected/a75a2281-4146-464f-bdda-8f221a98f52d-kube-api-access-kfstq\") pod \"monitoring-plugin-d5dcff7cf-bp5hj\" (UID: \"a75a2281-4146-464f-bdda-8f221a98f52d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" Jul 02 19:03:53.551410 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:53.551378 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a75a2281-4146-464f-bdda-8f221a98f52d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bp5hj\" (UID: \"a75a2281-4146-464f-bdda-8f221a98f52d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" Jul 02 19:03:53.554086 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:53.554045 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a75a2281-4146-464f-bdda-8f221a98f52d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bp5hj\" (UID: \"a75a2281-4146-464f-bdda-8f221a98f52d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" Jul 02 19:03:53.765973 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:53.765930 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" Jul 02 19:03:55.335783 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:55.335751 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4e7167_ea2a_4cb7_9c30_b29a87cb07f3.slice/crio-998b2c02bc39bbb3040cc27600f252c27b0ddb98702fd905bcf0218ef1b03e08 WatchSource:0}: Error finding container 998b2c02bc39bbb3040cc27600f252c27b0ddb98702fd905bcf0218ef1b03e08: Status 404 returned error can't find the container with id 998b2c02bc39bbb3040cc27600f252c27b0ddb98702fd905bcf0218ef1b03e08 Jul 02 19:03:55.429890 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:55.428815 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c7bdd876f-4rccg" Jul 02 19:03:55.539100 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:55.538825 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" event={"ID":"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3","Type":"ContainerStarted","Data":"998b2c02bc39bbb3040cc27600f252c27b0ddb98702fd905bcf0218ef1b03e08"} Jul 02 19:03:55.541228 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:55.540779 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z6zb2" event={"ID":"7c305fe4-c346-4c57-b6c6-1dea6f663801","Type":"ContainerStarted","Data":"c25d541889171bc94202348dc5ebe3afc5ded7b0461c126e3790ffd239e0df0b"} Jul 02 19:03:55.541228 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:55.540898 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:03:55.562109 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:55.562057 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj"] Jul 02 19:03:55.562860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:55.562759 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z6zb2" podStartSLOduration=66.539643557 podStartE2EDuration="1m12.562739427s" totalImagesPullingTime="6.02309587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:55.562556728 +0000 UTC m=+72.853893628" watchObservedRunningTime="2026-07-02 19:03:55.562739427 +0000 UTC m=+72.854076328" Jul 02 19:03:55.567994 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:03:55.567952 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda75a2281_4146_464f_bdda_8f221a98f52d.slice/crio-eb3a03de9b04780fe16a68381920f274249cffc14c7c7a33d3656ae0073b96fe WatchSource:0}: Error finding container eb3a03de9b04780fe16a68381920f274249cffc14c7c7a33d3656ae0073b96fe: Status 404 returned error can't find the container with id eb3a03de9b04780fe16a68381920f274249cffc14c7c7a33d3656ae0073b96fe Jul 02 19:03:56.547359 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:56.547317 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8sprk" event={"ID":"9332d088-8ecc-4c56-a0d8-6d35d5acda58","Type":"ContainerStarted","Data":"86ed7ed3cc936e21da94843ac9303608ca20a90c27115ab5947b0d3a48fe7de7"} Jul 02 19:03:56.548596 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:56.548572 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" event={"ID":"a75a2281-4146-464f-bdda-8f221a98f52d","Type":"ContainerStarted","Data":"eb3a03de9b04780fe16a68381920f274249cffc14c7c7a33d3656ae0073b96fe"} Jul 02 19:03:56.579299 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:56.579240 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8sprk" podStartSLOduration=67.518194711 podStartE2EDuration="1m13.579224454s" totalImagesPullingTime="6.061029743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:56.576550289 +0000 UTC m=+73.867887188" watchObservedRunningTime="2026-07-02 19:03:56.579224454 +0000 UTC m=+73.870561353" Jul 02 19:03:58.557740 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:58.557704 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" event={"ID":"1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3","Type":"ContainerStarted","Data":"c0e7e1d1526ea0cb90dcd55c92ba48502b8b1c52f52b7ea2b593af48cb67e934"} Jul 02 19:03:58.560801 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:58.560770 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50","Type":"ContainerStarted","Data":"b3c352857967b689b36830060cc51cf1e88154dc8ce95f657c3b766cc9c9aa90"} Jul 02 19:03:58.560943 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:58.560847 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50","Type":"ContainerStarted","Data":"15965e752aa89dfb16b66b42928bf7453c8063f2a6ce5e8c40cfc7747b408fdb"} Jul 02 19:03:58.560943 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:58.560885 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50","Type":"ContainerStarted","Data":"59d48ec6666fc93c87950e18410ef8cbb4cba97c3abe9bee8c02d8a656a52c89"} Jul 02 19:03:58.560943 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:58.560920 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50","Type":"ContainerStarted","Data":"1874f115200ec4eceb485190c98f3fb6f9aac9e309846b65b48521af49048cfb"} Jul 02 19:03:58.561060 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:58.560948 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50","Type":"ContainerStarted","Data":"293fb1b22a374a1411f1c0407f466ddc5ae7d92fe30c3855e68adc73550b396b"} Jul 02 19:03:58.562226 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:58.562204 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" event={"ID":"a75a2281-4146-464f-bdda-8f221a98f52d","Type":"ContainerStarted","Data":"ed4c453960bca46e3092d442fcfbd18ee48ee73b7e31d9a56285b0361d60fdfe"} Jul 02 19:03:58.562463 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:58.562445 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" Jul 02 19:03:58.567176 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:58.567158 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" Jul 02 19:03:58.575424 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:58.575380 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" podStartSLOduration=4.143331821 podStartE2EDuration="6.575367381s" totalImagesPullingTime="2.43203556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:58.573651085 +0000 UTC m=+75.864988009" watchObservedRunningTime="2026-07-02 19:03:58.575367381 +0000 UTC m=+75.866704339" Jul 02 19:03:58.592727 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:58.592655 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bp5hj" podStartSLOduration=4.348739706 podStartE2EDuration="6.592637878s" totalImagesPullingTime="2.243898172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:58.59204836 +0000 UTC m=+75.883385262" watchObservedRunningTime="2026-07-02 19:03:58.592637878 +0000 UTC m=+75.883974778" Jul 02 19:03:59.569105 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:59.569057 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"241b3746-c7e5-43d4-9e38-cc0b2d7a9b50","Type":"ContainerStarted","Data":"0d77a81a623d3b94ee6d157d2235dfeb4992be52aa4ef5f6d2b70fd7271277d4"} Jul 02 19:03:59.616815 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:03:59.616764 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.936677092 podStartE2EDuration="10.616746432s" totalImagesPullingTime="7.66332585s" totalInitContainerRuntime="16.74349ms" isStatefulPod=false podCreationTimestamp="2026-07-02 19:03:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:03:59.616179753 +0000 UTC m=+76.907516654" watchObservedRunningTime="2026-07-02 19:03:59.616746432 +0000 UTC m=+76.908083333" Jul 02 19:04:12.810986 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:04:12.810861 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:04:12.810986 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:04:12.810918 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:04:26.551261 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:04:26.551231 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z6zb2" Jul 02 19:04:32.816028 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:04:32.815997 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:04:32.819826 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:04:32.819801 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5bd5947889-lfh8v" Jul 02 19:05:12.418678 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.418639 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-785858fb6-bvr9s"] Jul 02 19:05:12.420922 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.420906 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.424233 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.424212 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 02 19:05:12.424568 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.424552 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 02 19:05:12.424749 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.424735 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 02 19:05:12.425098 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.425081 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 02 19:05:12.425176 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.425141 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 02 19:05:12.425350 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.425335 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-xtbmd\"" Jul 02 19:05:12.434963 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.434940 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 02 19:05:12.458503 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.458474 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-785858fb6-bvr9s"] Jul 02 19:05:12.586025 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.585973 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10732671-4620-4659-a188-71fefbcf5333-metrics-client-ca\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.586025 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.586035 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-secret-telemeter-client\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.586316 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.586064 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10732671-4620-4659-a188-71fefbcf5333-serving-certs-ca-bundle\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.586316 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.586127 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10732671-4620-4659-a188-71fefbcf5333-telemeter-trusted-ca-bundle\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.586316 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.586198 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.586316 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.586226 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl8xr\" (UniqueName: \"kubernetes.io/projected/10732671-4620-4659-a188-71fefbcf5333-kube-api-access-nl8xr\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.586316 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.586263 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-federate-client-tls\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.586498 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.586341 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-telemeter-client-tls\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.687448 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.687347 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-federate-client-tls\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.687448 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.687406 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-telemeter-client-tls\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.687448 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.687443 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10732671-4620-4659-a188-71fefbcf5333-metrics-client-ca\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.687695 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.687471 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-secret-telemeter-client\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.687695 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.687498 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10732671-4620-4659-a188-71fefbcf5333-serving-certs-ca-bundle\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.687695 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.687587 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10732671-4620-4659-a188-71fefbcf5333-telemeter-trusted-ca-bundle\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.687695 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.687641 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.687695 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.687662 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl8xr\" (UniqueName: \"kubernetes.io/projected/10732671-4620-4659-a188-71fefbcf5333-kube-api-access-nl8xr\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.688602 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.688571 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10732671-4620-4659-a188-71fefbcf5333-metrics-client-ca\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.688753 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.688608 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10732671-4620-4659-a188-71fefbcf5333-serving-certs-ca-bundle\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.688753 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.688684 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10732671-4620-4659-a188-71fefbcf5333-telemeter-trusted-ca-bundle\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.690288 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.690262 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-secret-telemeter-client\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.690414 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.690262 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.690581 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.690558 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-federate-client-tls\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.690637 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.690561 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10732671-4620-4659-a188-71fefbcf5333-telemeter-client-tls\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.695576 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.695555 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl8xr\" (UniqueName: \"kubernetes.io/projected/10732671-4620-4659-a188-71fefbcf5333-kube-api-access-nl8xr\") pod \"telemeter-client-785858fb6-bvr9s\" (UID: \"10732671-4620-4659-a188-71fefbcf5333\") " pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.731238 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.731201 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" Jul 02 19:05:12.868936 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:12.868895 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-785858fb6-bvr9s"] Jul 02 19:05:12.872305 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:05:12.872280 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10732671_4620_4659_a188_71fefbcf5333.slice/crio-8912b7669638da509b69b4929dbfb47c42fdcc2683fd43f74676fff70e09d433 WatchSource:0}: Error finding container 8912b7669638da509b69b4929dbfb47c42fdcc2683fd43f74676fff70e09d433: Status 404 returned error can't find the container with id 8912b7669638da509b69b4929dbfb47c42fdcc2683fd43f74676fff70e09d433 Jul 02 19:05:13.848937 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:13.848898 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" event={"ID":"10732671-4620-4659-a188-71fefbcf5333","Type":"ContainerStarted","Data":"8912b7669638da509b69b4929dbfb47c42fdcc2683fd43f74676fff70e09d433"} Jul 02 19:05:14.854082 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:14.854041 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" event={"ID":"10732671-4620-4659-a188-71fefbcf5333","Type":"ContainerStarted","Data":"19837674b43f2cf24bb2c5e207321905b9f90b85dbac89af8bf97cec559eaf2d"} Jul 02 19:05:14.854450 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:14.854124 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" event={"ID":"10732671-4620-4659-a188-71fefbcf5333","Type":"ContainerStarted","Data":"a980de07e2b0ca8375dd1cab13f837846c3984b4f69caf9a1b2943213accb640"} Jul 02 19:05:14.854450 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:14.854153 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" event={"ID":"10732671-4620-4659-a188-71fefbcf5333","Type":"ContainerStarted","Data":"1c93c29381e89c2019358f77f267c7d5d2288a34175cb28cdd67048c4dde25ec"} Jul 02 19:05:14.888522 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:05:14.888466 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-785858fb6-bvr9s" podStartSLOduration=1.37609816 podStartE2EDuration="2.88844904s" totalImagesPullingTime="1.51235088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:14.88638416 +0000 UTC m=+152.177721062" watchObservedRunningTime="2026-07-02 19:05:14.88844904 +0000 UTC m=+152.179785967" Jul 02 19:06:20.246793 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.246763 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc"] Jul 02 19:06:20.250392 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.250369 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:20.252848 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.252827 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9pj28\"" Jul 02 19:06:20.253061 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.253037 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 19:06:20.253598 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.253581 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 19:06:20.258385 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.258362 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc"] Jul 02 19:06:20.322592 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.322553 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:20.322765 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.322613 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl5w9\" (UniqueName: \"kubernetes.io/projected/597195ef-a1cc-46a6-b694-f99f27496757-kube-api-access-vl5w9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:20.322765 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.322686 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:20.423095 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.423035 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl5w9\" (UniqueName: \"kubernetes.io/projected/597195ef-a1cc-46a6-b694-f99f27496757-kube-api-access-vl5w9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:20.423268 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.423105 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:20.423268 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.423139 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:20.423473 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.423458 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:20.423522 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.423501 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:20.431203 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.431172 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl5w9\" (UniqueName: \"kubernetes.io/projected/597195ef-a1cc-46a6-b694-f99f27496757-kube-api-access-vl5w9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:20.560382 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.560350 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:20.693834 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:20.691151 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc"] Jul 02 19:06:20.696831 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:06:20.696801 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod597195ef_a1cc_46a6_b694_f99f27496757.slice/crio-15ea1237228cd8429b0a5cf88e50609004e764ee0dc2d9d34be15b9614c7cd73 WatchSource:0}: Error finding container 15ea1237228cd8429b0a5cf88e50609004e764ee0dc2d9d34be15b9614c7cd73: Status 404 returned error can't find the container with id 15ea1237228cd8429b0a5cf88e50609004e764ee0dc2d9d34be15b9614c7cd73 Jul 02 19:06:21.086050 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:21.086016 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" event={"ID":"597195ef-a1cc-46a6-b694-f99f27496757","Type":"ContainerStarted","Data":"15ea1237228cd8429b0a5cf88e50609004e764ee0dc2d9d34be15b9614c7cd73"} Jul 02 19:06:30.121649 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:30.121611 2571 generic.go:356] "Generic (PLEG): container finished" podID="597195ef-a1cc-46a6-b694-f99f27496757" containerID="eb0920047907323e895a4dff77583ad83ee0cb31dfef87c52d613092b68ff3f0" exitCode=0 Jul 02 19:06:30.121649 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:30.121653 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" event={"ID":"597195ef-a1cc-46a6-b694-f99f27496757","Type":"ContainerDied","Data":"eb0920047907323e895a4dff77583ad83ee0cb31dfef87c52d613092b68ff3f0"} Jul 02 19:06:32.131167 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:32.131135 2571 generic.go:356] "Generic (PLEG): container finished" podID="597195ef-a1cc-46a6-b694-f99f27496757" containerID="fc8b63443ef9e3043b6fdb6223067e3ab8bd608e481a3d8fa91f9dc61d1fafaa" exitCode=0 Jul 02 19:06:32.131620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:32.131173 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" event={"ID":"597195ef-a1cc-46a6-b694-f99f27496757","Type":"ContainerDied","Data":"fc8b63443ef9e3043b6fdb6223067e3ab8bd608e481a3d8fa91f9dc61d1fafaa"} Jul 02 19:06:41.170490 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:41.170426 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" event={"ID":"597195ef-a1cc-46a6-b694-f99f27496757","Type":"ContainerStarted","Data":"c6ec9d6ec330cac54ebfb0d58f012f218463ccc020679561e85a5595cd9b6092"} Jul 02 19:06:41.194234 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:41.194179 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" podStartSLOduration=1.660707231 podStartE2EDuration="21.194160169s" totalImagesPullingTime="19.533452938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:41.192840303 +0000 UTC m=+238.484177204" watchObservedRunningTime="2026-07-02 19:06:41.194160169 +0000 UTC m=+238.485497070" Jul 02 19:06:42.176141 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:42.176106 2571 generic.go:356] "Generic (PLEG): container finished" podID="597195ef-a1cc-46a6-b694-f99f27496757" containerID="c6ec9d6ec330cac54ebfb0d58f012f218463ccc020679561e85a5595cd9b6092" exitCode=0 Jul 02 19:06:42.176531 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:42.176164 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" event={"ID":"597195ef-a1cc-46a6-b694-f99f27496757","Type":"ContainerDied","Data":"c6ec9d6ec330cac54ebfb0d58f012f218463ccc020679561e85a5595cd9b6092"} Jul 02 19:06:43.355854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:43.355821 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:43.429281 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:43.429247 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-bundle\") pod \"597195ef-a1cc-46a6-b694-f99f27496757\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " Jul 02 19:06:43.429460 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:43.429308 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-util\") pod \"597195ef-a1cc-46a6-b694-f99f27496757\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " Jul 02 19:06:43.429460 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:43.429343 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vl5w9\" (UniqueName: \"kubernetes.io/projected/597195ef-a1cc-46a6-b694-f99f27496757-kube-api-access-vl5w9\") pod \"597195ef-a1cc-46a6-b694-f99f27496757\" (UID: \"597195ef-a1cc-46a6-b694-f99f27496757\") " Jul 02 19:06:43.429941 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:43.429905 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-bundle" (OuterVolumeSpecName: "bundle") pod "597195ef-a1cc-46a6-b694-f99f27496757" (UID: "597195ef-a1cc-46a6-b694-f99f27496757"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:06:43.431519 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:43.431480 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/597195ef-a1cc-46a6-b694-f99f27496757-kube-api-access-vl5w9" (OuterVolumeSpecName: "kube-api-access-vl5w9") pod "597195ef-a1cc-46a6-b694-f99f27496757" (UID: "597195ef-a1cc-46a6-b694-f99f27496757"). InnerVolumeSpecName "kube-api-access-vl5w9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:06:43.433497 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:43.433476 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-util" (OuterVolumeSpecName: "util") pod "597195ef-a1cc-46a6-b694-f99f27496757" (UID: "597195ef-a1cc-46a6-b694-f99f27496757"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:06:43.530094 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:43.530036 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-util\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:06:43.530094 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:43.530064 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl5w9\" (UniqueName: \"kubernetes.io/projected/597195ef-a1cc-46a6-b694-f99f27496757-kube-api-access-vl5w9\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:06:43.530094 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:43.530095 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/597195ef-a1cc-46a6-b694-f99f27496757-bundle\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:06:44.185232 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:44.185203 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" Jul 02 19:06:44.185405 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:44.185202 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ksjmc" event={"ID":"597195ef-a1cc-46a6-b694-f99f27496757","Type":"ContainerDied","Data":"15ea1237228cd8429b0a5cf88e50609004e764ee0dc2d9d34be15b9614c7cd73"} Jul 02 19:06:44.185405 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:44.185327 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15ea1237228cd8429b0a5cf88e50609004e764ee0dc2d9d34be15b9614c7cd73" Jul 02 19:06:48.545033 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.544999 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64"] Jul 02 19:06:48.545494 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.545313 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="597195ef-a1cc-46a6-b694-f99f27496757" containerName="pull" Jul 02 19:06:48.545494 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.545324 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="597195ef-a1cc-46a6-b694-f99f27496757" containerName="pull" Jul 02 19:06:48.545494 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.545333 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="597195ef-a1cc-46a6-b694-f99f27496757" containerName="extract" Jul 02 19:06:48.545494 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.545338 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="597195ef-a1cc-46a6-b694-f99f27496757" containerName="extract" Jul 02 19:06:48.545494 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.545348 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="597195ef-a1cc-46a6-b694-f99f27496757" containerName="util" Jul 02 19:06:48.545494 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.545354 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="597195ef-a1cc-46a6-b694-f99f27496757" containerName="util" Jul 02 19:06:48.545494 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.545400 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="597195ef-a1cc-46a6-b694-f99f27496757" containerName="extract" Jul 02 19:06:48.547812 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.547794 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64" Jul 02 19:06:48.550510 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.550487 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:06:48.550944 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.550924 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 02 19:06:48.551111 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.551097 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-wnndr\"" Jul 02 19:06:48.563661 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.563629 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64"] Jul 02 19:06:48.670882 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.670842 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/535c3067-19f9-4fb1-a1a8-0a8e0762b52b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bsh64\" (UID: \"535c3067-19f9-4fb1-a1a8-0a8e0762b52b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64" Jul 02 19:06:48.671057 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.670901 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrsz8\" (UniqueName: \"kubernetes.io/projected/535c3067-19f9-4fb1-a1a8-0a8e0762b52b-kube-api-access-zrsz8\") pod \"cert-manager-operator-controller-manager-55c444bf86-bsh64\" (UID: \"535c3067-19f9-4fb1-a1a8-0a8e0762b52b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64" Jul 02 19:06:48.772127 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.772053 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/535c3067-19f9-4fb1-a1a8-0a8e0762b52b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bsh64\" (UID: \"535c3067-19f9-4fb1-a1a8-0a8e0762b52b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64" Jul 02 19:06:48.772289 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.772162 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrsz8\" (UniqueName: \"kubernetes.io/projected/535c3067-19f9-4fb1-a1a8-0a8e0762b52b-kube-api-access-zrsz8\") pod \"cert-manager-operator-controller-manager-55c444bf86-bsh64\" (UID: \"535c3067-19f9-4fb1-a1a8-0a8e0762b52b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64" Jul 02 19:06:48.772457 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.772429 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/535c3067-19f9-4fb1-a1a8-0a8e0762b52b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bsh64\" (UID: \"535c3067-19f9-4fb1-a1a8-0a8e0762b52b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64" Jul 02 19:06:48.780658 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.780626 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrsz8\" (UniqueName: \"kubernetes.io/projected/535c3067-19f9-4fb1-a1a8-0a8e0762b52b-kube-api-access-zrsz8\") pod \"cert-manager-operator-controller-manager-55c444bf86-bsh64\" (UID: \"535c3067-19f9-4fb1-a1a8-0a8e0762b52b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64" Jul 02 19:06:48.857969 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:48.857867 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64" Jul 02 19:06:49.008334 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:49.008303 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64"] Jul 02 19:06:49.011246 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:06:49.011220 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod535c3067_19f9_4fb1_a1a8_0a8e0762b52b.slice/crio-e54baa44932ecefbe90841058e3f431ead1af0ba8f77d884ef38cb85af2e777d WatchSource:0}: Error finding container e54baa44932ecefbe90841058e3f431ead1af0ba8f77d884ef38cb85af2e777d: Status 404 returned error can't find the container with id e54baa44932ecefbe90841058e3f431ead1af0ba8f77d884ef38cb85af2e777d Jul 02 19:06:49.204386 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:49.204302 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64" event={"ID":"535c3067-19f9-4fb1-a1a8-0a8e0762b52b","Type":"ContainerStarted","Data":"e54baa44932ecefbe90841058e3f431ead1af0ba8f77d884ef38cb85af2e777d"} Jul 02 19:06:58.242032 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.241995 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64" event={"ID":"535c3067-19f9-4fb1-a1a8-0a8e0762b52b","Type":"ContainerStarted","Data":"2fe18db9650902d4807f51d178143b7e31ae3d4e8f0064e37c5d0d130c0562dd"} Jul 02 19:06:58.268871 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.268815 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bsh64" podStartSLOduration=1.8814959839999998 podStartE2EDuration="10.268797071s" totalImagesPullingTime="8.387301087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:58.266760959 +0000 UTC m=+255.558097863" watchObservedRunningTime="2026-07-02 19:06:58.268797071 +0000 UTC m=+255.560133970" Jul 02 19:06:58.429380 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.429344 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2pcrt"] Jul 02 19:06:58.431443 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.431424 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" Jul 02 19:06:58.433731 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.433716 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 19:06:58.433804 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.433790 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 19:06:58.444097 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.444057 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2pcrt"] Jul 02 19:06:58.561059 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.561020 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcx25\" (UniqueName: \"kubernetes.io/projected/a67aed40-6b51-4d69-be51-399b5640c509-kube-api-access-vcx25\") pod \"cert-manager-webhook-6486ffd997-2pcrt\" (UID: \"a67aed40-6b51-4d69-be51-399b5640c509\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" Jul 02 19:06:58.561253 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.561111 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a67aed40-6b51-4d69-be51-399b5640c509-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2pcrt\" (UID: \"a67aed40-6b51-4d69-be51-399b5640c509\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" Jul 02 19:06:58.662241 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.662210 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcx25\" (UniqueName: \"kubernetes.io/projected/a67aed40-6b51-4d69-be51-399b5640c509-kube-api-access-vcx25\") pod \"cert-manager-webhook-6486ffd997-2pcrt\" (UID: \"a67aed40-6b51-4d69-be51-399b5640c509\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" Jul 02 19:06:58.662410 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.662261 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a67aed40-6b51-4d69-be51-399b5640c509-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2pcrt\" (UID: \"a67aed40-6b51-4d69-be51-399b5640c509\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" Jul 02 19:06:58.681144 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.681115 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcx25\" (UniqueName: \"kubernetes.io/projected/a67aed40-6b51-4d69-be51-399b5640c509-kube-api-access-vcx25\") pod \"cert-manager-webhook-6486ffd997-2pcrt\" (UID: \"a67aed40-6b51-4d69-be51-399b5640c509\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" Jul 02 19:06:58.685015 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.684988 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a67aed40-6b51-4d69-be51-399b5640c509-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2pcrt\" (UID: \"a67aed40-6b51-4d69-be51-399b5640c509\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" Jul 02 19:06:58.752757 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.752717 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" Jul 02 19:06:58.894698 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:58.893854 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2pcrt"] Jul 02 19:06:59.246721 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.246637 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" event={"ID":"a67aed40-6b51-4d69-be51-399b5640c509","Type":"ContainerStarted","Data":"d4e99748ff7a8a3b55e8c4c2e012fe56e5f10c7912aea6598caab8e96e23d7b1"} Jul 02 19:06:59.774692 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.774656 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd"] Jul 02 19:06:59.777417 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.777391 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:06:59.779812 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.779767 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 19:06:59.779942 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.779859 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9pj28\"" Jul 02 19:06:59.780645 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.780623 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 19:06:59.786820 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.786749 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd"] Jul 02 19:06:59.872423 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.872383 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:06:59.872617 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.872432 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrhdk\" (UniqueName: \"kubernetes.io/projected/3b8c6b4a-0119-47a0-9680-0377602542c1-kube-api-access-vrhdk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:06:59.872617 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.872473 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:06:59.973847 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.973804 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:06:59.974014 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.973856 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrhdk\" (UniqueName: \"kubernetes.io/projected/3b8c6b4a-0119-47a0-9680-0377602542c1-kube-api-access-vrhdk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:06:59.974014 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.973897 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:06:59.974293 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.974273 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:06:59.974351 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.974307 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:06:59.982238 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:06:59.982211 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrhdk\" (UniqueName: \"kubernetes.io/projected/3b8c6b4a-0119-47a0-9680-0377602542c1-kube-api-access-vrhdk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:07:00.090378 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:00.090283 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:07:00.244610 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:00.244507 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd"] Jul 02 19:07:00.247524 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:07:00.247472 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8c6b4a_0119_47a0_9680_0377602542c1.slice/crio-16b2bd8a178415819c7656fb9f23d8a6f296ee5874a0e3af5e92fea204bd7c9d WatchSource:0}: Error finding container 16b2bd8a178415819c7656fb9f23d8a6f296ee5874a0e3af5e92fea204bd7c9d: Status 404 returned error can't find the container with id 16b2bd8a178415819c7656fb9f23d8a6f296ee5874a0e3af5e92fea204bd7c9d Jul 02 19:07:01.258290 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:01.258196 2571 generic.go:356] "Generic (PLEG): container finished" podID="3b8c6b4a-0119-47a0-9680-0377602542c1" containerID="52f56c61d2254cbaf380f3ad8ad41958ff6c2949740d775a5278676e05388478" exitCode=0 Jul 02 19:07:01.258290 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:01.258270 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" event={"ID":"3b8c6b4a-0119-47a0-9680-0377602542c1","Type":"ContainerDied","Data":"52f56c61d2254cbaf380f3ad8ad41958ff6c2949740d775a5278676e05388478"} Jul 02 19:07:01.258789 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:01.258316 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" event={"ID":"3b8c6b4a-0119-47a0-9680-0377602542c1","Type":"ContainerStarted","Data":"16b2bd8a178415819c7656fb9f23d8a6f296ee5874a0e3af5e92fea204bd7c9d"} Jul 02 19:07:02.264238 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:02.264194 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" event={"ID":"a67aed40-6b51-4d69-be51-399b5640c509","Type":"ContainerStarted","Data":"aa5790050c8b968cc03cb58fa1bdaf829db3917f8ec7357b272c382977961d87"} Jul 02 19:07:02.264630 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:02.264255 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" Jul 02 19:07:02.283748 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:02.283691 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" podStartSLOduration=1.696700464 podStartE2EDuration="4.283673257s" totalImagesPullingTime="2.586972793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:07:02.282578489 +0000 UTC m=+259.573915392" watchObservedRunningTime="2026-07-02 19:07:02.283673257 +0000 UTC m=+259.575010156" Jul 02 19:07:04.906958 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:04.906876 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s62vn"] Jul 02 19:07:04.909128 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:04.909111 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s62vn" Jul 02 19:07:04.911524 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:04.911504 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8lplz\"" Jul 02 19:07:04.918018 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:04.917988 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s62vn"] Jul 02 19:07:04.922841 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:04.922814 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hq28\" (UniqueName: \"kubernetes.io/projected/62c8601d-a986-48e3-bbae-6b4d48750638-kube-api-access-4hq28\") pod \"cert-manager-cainjector-559cdd8c87-s62vn\" (UID: \"62c8601d-a986-48e3-bbae-6b4d48750638\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s62vn" Jul 02 19:07:04.922981 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:04.922848 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62c8601d-a986-48e3-bbae-6b4d48750638-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s62vn\" (UID: \"62c8601d-a986-48e3-bbae-6b4d48750638\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s62vn" Jul 02 19:07:05.024306 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:05.024275 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hq28\" (UniqueName: \"kubernetes.io/projected/62c8601d-a986-48e3-bbae-6b4d48750638-kube-api-access-4hq28\") pod \"cert-manager-cainjector-559cdd8c87-s62vn\" (UID: \"62c8601d-a986-48e3-bbae-6b4d48750638\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s62vn" Jul 02 19:07:05.024306 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:05.024312 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62c8601d-a986-48e3-bbae-6b4d48750638-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s62vn\" (UID: \"62c8601d-a986-48e3-bbae-6b4d48750638\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s62vn" Jul 02 19:07:05.034611 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:05.034575 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62c8601d-a986-48e3-bbae-6b4d48750638-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s62vn\" (UID: \"62c8601d-a986-48e3-bbae-6b4d48750638\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s62vn" Jul 02 19:07:05.034762 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:05.034732 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hq28\" (UniqueName: \"kubernetes.io/projected/62c8601d-a986-48e3-bbae-6b4d48750638-kube-api-access-4hq28\") pod \"cert-manager-cainjector-559cdd8c87-s62vn\" (UID: \"62c8601d-a986-48e3-bbae-6b4d48750638\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s62vn" Jul 02 19:07:05.218885 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:05.218804 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s62vn" Jul 02 19:07:05.348216 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:05.348051 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s62vn"] Jul 02 19:07:05.351041 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:07:05.351007 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62c8601d_a986_48e3_bbae_6b4d48750638.slice/crio-56643dfd749d40c936277b153131868ca0d425301a2ab70d0bb9e487b9b52226 WatchSource:0}: Error finding container 56643dfd749d40c936277b153131868ca0d425301a2ab70d0bb9e487b9b52226: Status 404 returned error can't find the container with id 56643dfd749d40c936277b153131868ca0d425301a2ab70d0bb9e487b9b52226 Jul 02 19:07:06.282988 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:06.282951 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s62vn" event={"ID":"62c8601d-a986-48e3-bbae-6b4d48750638","Type":"ContainerStarted","Data":"6377a3749d9f5fbf56765f97418085f961093509175aa94edda1f01fab573fcf"} Jul 02 19:07:06.283372 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:06.283002 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s62vn" event={"ID":"62c8601d-a986-48e3-bbae-6b4d48750638","Type":"ContainerStarted","Data":"56643dfd749d40c936277b153131868ca0d425301a2ab70d0bb9e487b9b52226"} Jul 02 19:07:06.300718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:06.300664 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s62vn" podStartSLOduration=2.300648953 podStartE2EDuration="2.300648953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:07:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:07:06.299225933 +0000 UTC m=+263.590562846" watchObservedRunningTime="2026-07-02 19:07:06.300648953 +0000 UTC m=+263.591985853" Jul 02 19:07:08.270583 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:08.270557 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2pcrt" Jul 02 19:07:15.321713 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:15.321681 2571 generic.go:356] "Generic (PLEG): container finished" podID="3b8c6b4a-0119-47a0-9680-0377602542c1" containerID="9b1cb074a5b9b76b973e0def3e00ad9997a1b52ac71480688963721045c0ae2e" exitCode=0 Jul 02 19:07:15.322226 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:15.321750 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" event={"ID":"3b8c6b4a-0119-47a0-9680-0377602542c1","Type":"ContainerDied","Data":"9b1cb074a5b9b76b973e0def3e00ad9997a1b52ac71480688963721045c0ae2e"} Jul 02 19:07:16.328241 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:16.328203 2571 generic.go:356] "Generic (PLEG): container finished" podID="3b8c6b4a-0119-47a0-9680-0377602542c1" containerID="2aa9ecc6aff7247933e9b927d49369cdb163f7029ac20f980ae615c49802e425" exitCode=0 Jul 02 19:07:16.328619 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:16.328267 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" event={"ID":"3b8c6b4a-0119-47a0-9680-0377602542c1","Type":"ContainerDied","Data":"2aa9ecc6aff7247933e9b927d49369cdb163f7029ac20f980ae615c49802e425"} Jul 02 19:07:17.452347 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:17.452318 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:07:17.533247 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:17.533215 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrhdk\" (UniqueName: \"kubernetes.io/projected/3b8c6b4a-0119-47a0-9680-0377602542c1-kube-api-access-vrhdk\") pod \"3b8c6b4a-0119-47a0-9680-0377602542c1\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " Jul 02 19:07:17.533431 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:17.533301 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-util\") pod \"3b8c6b4a-0119-47a0-9680-0377602542c1\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " Jul 02 19:07:17.533431 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:17.533332 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-bundle\") pod \"3b8c6b4a-0119-47a0-9680-0377602542c1\" (UID: \"3b8c6b4a-0119-47a0-9680-0377602542c1\") " Jul 02 19:07:17.533814 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:17.533777 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-bundle" (OuterVolumeSpecName: "bundle") pod "3b8c6b4a-0119-47a0-9680-0377602542c1" (UID: "3b8c6b4a-0119-47a0-9680-0377602542c1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:07:17.535509 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:17.535483 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b8c6b4a-0119-47a0-9680-0377602542c1-kube-api-access-vrhdk" (OuterVolumeSpecName: "kube-api-access-vrhdk") pod "3b8c6b4a-0119-47a0-9680-0377602542c1" (UID: "3b8c6b4a-0119-47a0-9680-0377602542c1"). InnerVolumeSpecName "kube-api-access-vrhdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:07:17.540124 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:17.540098 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-util" (OuterVolumeSpecName: "util") pod "3b8c6b4a-0119-47a0-9680-0377602542c1" (UID: "3b8c6b4a-0119-47a0-9680-0377602542c1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:07:17.634412 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:17.634318 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-util\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:07:17.634412 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:17.634350 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b8c6b4a-0119-47a0-9680-0377602542c1-bundle\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:07:17.634412 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:17.634360 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrhdk\" (UniqueName: \"kubernetes.io/projected/3b8c6b4a-0119-47a0-9680-0377602542c1-kube-api-access-vrhdk\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:07:18.337398 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:18.337364 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" event={"ID":"3b8c6b4a-0119-47a0-9680-0377602542c1","Type":"ContainerDied","Data":"16b2bd8a178415819c7656fb9f23d8a6f296ee5874a0e3af5e92fea204bd7c9d"} Jul 02 19:07:18.337398 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:18.337393 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngtcd" Jul 02 19:07:18.337596 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:18.337412 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16b2bd8a178415819c7656fb9f23d8a6f296ee5874a0e3af5e92fea204bd7c9d" Jul 02 19:07:19.064868 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.064831 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5vhs9"] Jul 02 19:07:19.065251 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.065172 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b8c6b4a-0119-47a0-9680-0377602542c1" containerName="util" Jul 02 19:07:19.065251 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.065184 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b8c6b4a-0119-47a0-9680-0377602542c1" containerName="util" Jul 02 19:07:19.065251 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.065193 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b8c6b4a-0119-47a0-9680-0377602542c1" containerName="pull" Jul 02 19:07:19.065251 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.065198 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b8c6b4a-0119-47a0-9680-0377602542c1" containerName="pull" Jul 02 19:07:19.065251 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.065213 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b8c6b4a-0119-47a0-9680-0377602542c1" containerName="extract" Jul 02 19:07:19.065251 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.065219 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b8c6b4a-0119-47a0-9680-0377602542c1" containerName="extract" Jul 02 19:07:19.065495 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.065265 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b8c6b4a-0119-47a0-9680-0377602542c1" containerName="extract" Jul 02 19:07:19.067189 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.067173 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5vhs9" Jul 02 19:07:19.069246 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.069223 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-z74s9\"" Jul 02 19:07:19.076160 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.076137 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5vhs9"] Jul 02 19:07:19.148745 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.148707 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7-bound-sa-token\") pod \"cert-manager-55b4d954d5-5vhs9\" (UID: \"36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7\") " pod="cert-manager/cert-manager-55b4d954d5-5vhs9" Jul 02 19:07:19.148912 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.148792 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4qj5\" (UniqueName: \"kubernetes.io/projected/36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7-kube-api-access-d4qj5\") pod \"cert-manager-55b4d954d5-5vhs9\" (UID: \"36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7\") " pod="cert-manager/cert-manager-55b4d954d5-5vhs9" Jul 02 19:07:19.249397 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.249362 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4qj5\" (UniqueName: \"kubernetes.io/projected/36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7-kube-api-access-d4qj5\") pod \"cert-manager-55b4d954d5-5vhs9\" (UID: \"36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7\") " pod="cert-manager/cert-manager-55b4d954d5-5vhs9" Jul 02 19:07:19.249564 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.249416 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7-bound-sa-token\") pod \"cert-manager-55b4d954d5-5vhs9\" (UID: \"36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7\") " pod="cert-manager/cert-manager-55b4d954d5-5vhs9" Jul 02 19:07:19.257044 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.257010 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7-bound-sa-token\") pod \"cert-manager-55b4d954d5-5vhs9\" (UID: \"36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7\") " pod="cert-manager/cert-manager-55b4d954d5-5vhs9" Jul 02 19:07:19.257184 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.257167 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4qj5\" (UniqueName: \"kubernetes.io/projected/36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7-kube-api-access-d4qj5\") pod \"cert-manager-55b4d954d5-5vhs9\" (UID: \"36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7\") " pod="cert-manager/cert-manager-55b4d954d5-5vhs9" Jul 02 19:07:19.376561 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.376462 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5vhs9" Jul 02 19:07:19.503279 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:19.503250 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5vhs9"] Jul 02 19:07:19.505739 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:07:19.505709 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36f1ffc0_e63d_4165_8cd0_3a6f3a5832b7.slice/crio-a2b9acf10d41fa9ec176191539892325155eab9367e7da130045081a52c590d7 WatchSource:0}: Error finding container a2b9acf10d41fa9ec176191539892325155eab9367e7da130045081a52c590d7: Status 404 returned error can't find the container with id a2b9acf10d41fa9ec176191539892325155eab9367e7da130045081a52c590d7 Jul 02 19:07:20.347342 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:20.347298 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5vhs9" event={"ID":"36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7","Type":"ContainerStarted","Data":"ae400f5f97b961e6dbcdbb2f3d8f1f28cdc947fc3af223c576545ce280673e70"} Jul 02 19:07:20.347342 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:20.347349 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5vhs9" event={"ID":"36f1ffc0-e63d-4165-8cd0-3a6f3a5832b7","Type":"ContainerStarted","Data":"a2b9acf10d41fa9ec176191539892325155eab9367e7da130045081a52c590d7"} Jul 02 19:07:20.364230 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:20.364169 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-5vhs9" podStartSLOduration=1.36415218 podStartE2EDuration="1.36415218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:07:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:07:20.362491327 +0000 UTC m=+277.653828227" watchObservedRunningTime="2026-07-02 19:07:20.36415218 +0000 UTC m=+277.655489079" Jul 02 19:07:30.424566 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.424531 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk"] Jul 02 19:07:30.426909 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.426893 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:30.429344 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.429318 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 19:07:30.429457 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.429367 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 19:07:30.430403 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.430385 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9pj28\"" Jul 02 19:07:30.436866 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.436842 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk"] Jul 02 19:07:30.555147 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.555107 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:30.555345 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.555214 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:30.555345 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.555244 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpj6s\" (UniqueName: \"kubernetes.io/projected/7ddfaa43-1801-454a-9aea-6321b5b54dda-kube-api-access-wpj6s\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:30.656680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.656632 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:30.656680 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.656683 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpj6s\" (UniqueName: \"kubernetes.io/projected/7ddfaa43-1801-454a-9aea-6321b5b54dda-kube-api-access-wpj6s\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:30.657039 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.656737 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:30.657180 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.657161 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:30.657239 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.657187 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:30.665256 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.665229 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpj6s\" (UniqueName: \"kubernetes.io/projected/7ddfaa43-1801-454a-9aea-6321b5b54dda-kube-api-access-wpj6s\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:30.736877 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.736790 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:30.869026 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:30.868965 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk"] Jul 02 19:07:30.871188 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:07:30.871159 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ddfaa43_1801_454a_9aea_6321b5b54dda.slice/crio-380739d5e70675c23cf5231b8ed0e55428725a44114492be6dbffce7f9d33873 WatchSource:0}: Error finding container 380739d5e70675c23cf5231b8ed0e55428725a44114492be6dbffce7f9d33873: Status 404 returned error can't find the container with id 380739d5e70675c23cf5231b8ed0e55428725a44114492be6dbffce7f9d33873 Jul 02 19:07:31.394027 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:31.393990 2571 generic.go:356] "Generic (PLEG): container finished" podID="7ddfaa43-1801-454a-9aea-6321b5b54dda" containerID="115a7f22bed9213fdf2b139bb4a713155fde7a6b4f1581efb532dfae8a44f650" exitCode=0 Jul 02 19:07:31.394233 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:31.394095 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" event={"ID":"7ddfaa43-1801-454a-9aea-6321b5b54dda","Type":"ContainerDied","Data":"115a7f22bed9213fdf2b139bb4a713155fde7a6b4f1581efb532dfae8a44f650"} Jul 02 19:07:31.394233 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:31.394141 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" event={"ID":"7ddfaa43-1801-454a-9aea-6321b5b54dda","Type":"ContainerStarted","Data":"380739d5e70675c23cf5231b8ed0e55428725a44114492be6dbffce7f9d33873"} Jul 02 19:07:32.402612 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:32.402580 2571 generic.go:356] "Generic (PLEG): container finished" podID="7ddfaa43-1801-454a-9aea-6321b5b54dda" containerID="b753fc3801db7b082f509ef9f51a64174ffa19607caec6d7f61466327630effa" exitCode=0 Jul 02 19:07:32.403014 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:32.402672 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" event={"ID":"7ddfaa43-1801-454a-9aea-6321b5b54dda","Type":"ContainerDied","Data":"b753fc3801db7b082f509ef9f51a64174ffa19607caec6d7f61466327630effa"} Jul 02 19:07:33.409006 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:33.408974 2571 generic.go:356] "Generic (PLEG): container finished" podID="7ddfaa43-1801-454a-9aea-6321b5b54dda" containerID="0a64c3875bb0891573b6011002b61ce08045682a4337e965d6f105d1162abec1" exitCode=0 Jul 02 19:07:33.409489 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:33.409024 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" event={"ID":"7ddfaa43-1801-454a-9aea-6321b5b54dda","Type":"ContainerDied","Data":"0a64c3875bb0891573b6011002b61ce08045682a4337e965d6f105d1162abec1"} Jul 02 19:07:34.542688 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:34.542660 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:34.692565 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:34.692480 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpj6s\" (UniqueName: \"kubernetes.io/projected/7ddfaa43-1801-454a-9aea-6321b5b54dda-kube-api-access-wpj6s\") pod \"7ddfaa43-1801-454a-9aea-6321b5b54dda\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " Jul 02 19:07:34.692565 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:34.692523 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-util\") pod \"7ddfaa43-1801-454a-9aea-6321b5b54dda\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " Jul 02 19:07:34.692741 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:34.692673 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-bundle\") pod \"7ddfaa43-1801-454a-9aea-6321b5b54dda\" (UID: \"7ddfaa43-1801-454a-9aea-6321b5b54dda\") " Jul 02 19:07:34.693441 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:34.693401 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-bundle" (OuterVolumeSpecName: "bundle") pod "7ddfaa43-1801-454a-9aea-6321b5b54dda" (UID: "7ddfaa43-1801-454a-9aea-6321b5b54dda"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:07:34.694647 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:34.694618 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ddfaa43-1801-454a-9aea-6321b5b54dda-kube-api-access-wpj6s" (OuterVolumeSpecName: "kube-api-access-wpj6s") pod "7ddfaa43-1801-454a-9aea-6321b5b54dda" (UID: "7ddfaa43-1801-454a-9aea-6321b5b54dda"). InnerVolumeSpecName "kube-api-access-wpj6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:07:34.698312 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:34.698293 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-util" (OuterVolumeSpecName: "util") pod "7ddfaa43-1801-454a-9aea-6321b5b54dda" (UID: "7ddfaa43-1801-454a-9aea-6321b5b54dda"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:07:34.794266 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:34.794213 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpj6s\" (UniqueName: \"kubernetes.io/projected/7ddfaa43-1801-454a-9aea-6321b5b54dda-kube-api-access-wpj6s\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:07:34.794266 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:34.794258 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-util\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:07:34.794266 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:34.794269 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ddfaa43-1801-454a-9aea-6321b5b54dda-bundle\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:07:35.419712 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:35.419680 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" Jul 02 19:07:35.419712 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:35.419703 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzbqwk" event={"ID":"7ddfaa43-1801-454a-9aea-6321b5b54dda","Type":"ContainerDied","Data":"380739d5e70675c23cf5231b8ed0e55428725a44114492be6dbffce7f9d33873"} Jul 02 19:07:35.419913 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:35.419760 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="380739d5e70675c23cf5231b8ed0e55428725a44114492be6dbffce7f9d33873" Jul 02 19:07:43.110456 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:43.110428 2571 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 19:07:47.434605 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.434564 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf"] Jul 02 19:07:47.441551 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.434978 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ddfaa43-1801-454a-9aea-6321b5b54dda" containerName="pull" Jul 02 19:07:47.441551 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.434996 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ddfaa43-1801-454a-9aea-6321b5b54dda" containerName="pull" Jul 02 19:07:47.441551 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.435015 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ddfaa43-1801-454a-9aea-6321b5b54dda" containerName="extract" Jul 02 19:07:47.441551 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.435023 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ddfaa43-1801-454a-9aea-6321b5b54dda" containerName="extract" Jul 02 19:07:47.441551 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.435031 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ddfaa43-1801-454a-9aea-6321b5b54dda" containerName="util" Jul 02 19:07:47.441551 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.435037 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ddfaa43-1801-454a-9aea-6321b5b54dda" containerName="util" Jul 02 19:07:47.441551 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.435130 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ddfaa43-1801-454a-9aea-6321b5b54dda" containerName="extract" Jul 02 19:07:47.441968 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.441950 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.446801 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.446778 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 19:07:47.446934 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.446780 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 19:07:47.447503 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.447480 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hgcj5\"" Jul 02 19:07:47.447613 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.447544 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 19:07:47.447801 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.447780 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 19:07:47.461718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.461685 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf"] Jul 02 19:07:47.494378 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.494343 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b65de0f-2d80-4220-8186-a77feb3f847e-manifests\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.494564 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.494396 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlgdg\" (UniqueName: \"kubernetes.io/projected/1b65de0f-2d80-4220-8186-a77feb3f847e-kube-api-access-rlgdg\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.494564 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.494504 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b65de0f-2d80-4220-8186-a77feb3f847e-tmp\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.494564 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.494540 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b65de0f-2d80-4220-8186-a77feb3f847e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.494704 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.494572 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b65de0f-2d80-4220-8186-a77feb3f847e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.595726 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.595686 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b65de0f-2d80-4220-8186-a77feb3f847e-manifests\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.595933 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.595739 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlgdg\" (UniqueName: \"kubernetes.io/projected/1b65de0f-2d80-4220-8186-a77feb3f847e-kube-api-access-rlgdg\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.595933 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.595782 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b65de0f-2d80-4220-8186-a77feb3f847e-tmp\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.595933 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.595801 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b65de0f-2d80-4220-8186-a77feb3f847e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.595933 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.595823 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b65de0f-2d80-4220-8186-a77feb3f847e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.596212 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.596188 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b65de0f-2d80-4220-8186-a77feb3f847e-manifests\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.598175 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.598151 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b65de0f-2d80-4220-8186-a77feb3f847e-tmp\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.598286 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.598236 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b65de0f-2d80-4220-8186-a77feb3f847e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.598286 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.598270 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b65de0f-2d80-4220-8186-a77feb3f847e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.608907 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.608882 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlgdg\" (UniqueName: \"kubernetes.io/projected/1b65de0f-2d80-4220-8186-a77feb3f847e-kube-api-access-rlgdg\") pod \"opendatahub-operator-controller-manager-6cd5847cc4-x9xwf\" (UID: \"1b65de0f-2d80-4220-8186-a77feb3f847e\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.752225 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.752144 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:47.910107 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.910057 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf"] Jul 02 19:07:47.913346 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:07:47.913313 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b65de0f_2d80_4220_8186_a77feb3f847e.slice/crio-4aacb15ca43f19b671db49abf8bb1ae4e7ec2c34b19828cf98b46e34b1336eb1 WatchSource:0}: Error finding container 4aacb15ca43f19b671db49abf8bb1ae4e7ec2c34b19828cf98b46e34b1336eb1: Status 404 returned error can't find the container with id 4aacb15ca43f19b671db49abf8bb1ae4e7ec2c34b19828cf98b46e34b1336eb1 Jul 02 19:07:47.915310 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:47.915292 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:07:48.475150 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:48.475114 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" event={"ID":"1b65de0f-2d80-4220-8186-a77feb3f847e","Type":"ContainerStarted","Data":"4aacb15ca43f19b671db49abf8bb1ae4e7ec2c34b19828cf98b46e34b1336eb1"} Jul 02 19:07:51.490917 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:51.490833 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" event={"ID":"1b65de0f-2d80-4220-8186-a77feb3f847e","Type":"ContainerStarted","Data":"b8f3d9d67a40d702198d5429b2a80b91794b3719f1049705cc93b38a216caa07"} Jul 02 19:07:52.497890 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:52.497857 2571 generic.go:356] "Generic (PLEG): container finished" podID="1b65de0f-2d80-4220-8186-a77feb3f847e" containerID="b8f3d9d67a40d702198d5429b2a80b91794b3719f1049705cc93b38a216caa07" exitCode=0 Jul 02 19:07:52.498280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:52.497917 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" event={"ID":"1b65de0f-2d80-4220-8186-a77feb3f847e","Type":"ContainerDied","Data":"b8f3d9d67a40d702198d5429b2a80b91794b3719f1049705cc93b38a216caa07"} Jul 02 19:07:53.504161 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:53.504118 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" event={"ID":"1b65de0f-2d80-4220-8186-a77feb3f847e","Type":"ContainerStarted","Data":"fa69a9fe6ba864328bc4e0ce9f5337b5c4449062c2dd3198e37245c8d47175b8"} Jul 02 19:07:53.504573 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:53.504193 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:07:53.526727 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:07:53.526644 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" podStartSLOduration=1.838617977 podStartE2EDuration="6.526627024s" totalImagesPullingTime="3.592331987s" totalInitContainerRuntime="1.09567706s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:07:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:07:53.524578901 +0000 UTC m=+310.815915801" watchObservedRunningTime="2026-07-02 19:07:53.526627024 +0000 UTC m=+310.817963924" Jul 02 19:08:04.513032 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:04.512998 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6cd5847cc4-x9xwf" Jul 02 19:08:10.487180 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.487130 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t"] Jul 02 19:08:10.492084 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.492051 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-szvmv"] Jul 02 19:08:10.492240 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.492166 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:10.495548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.495440 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:10.496705 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.496685 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 19:08:10.497047 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.496897 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 19:08:10.497311 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.497273 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-p84kw\"" Jul 02 19:08:10.497383 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.497310 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 19:08:10.497445 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.497431 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 02 19:08:10.498153 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.498136 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sgt2g\"" Jul 02 19:08:10.501544 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.501524 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 19:08:10.511667 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.511643 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t"] Jul 02 19:08:10.513295 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.513267 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-szvmv"] Jul 02 19:08:10.598335 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.598303 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xczt\" (UniqueName: \"kubernetes.io/projected/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-kube-api-access-7xczt\") pod \"kube-auth-proxy-7656b89f9d-ccn4t\" (UID: \"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5\") " pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:10.598518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.598348 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51107ee4-db1f-4dfc-b67c-f43269462379-cert\") pod \"odh-model-controller-7998c67dc4-szvmv\" (UID: \"51107ee4-db1f-4dfc-b67c-f43269462379\") " pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:10.598518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.598426 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p957\" (UniqueName: \"kubernetes.io/projected/51107ee4-db1f-4dfc-b67c-f43269462379-kube-api-access-8p957\") pod \"odh-model-controller-7998c67dc4-szvmv\" (UID: \"51107ee4-db1f-4dfc-b67c-f43269462379\") " pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:10.598518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.598451 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-tls-certs\") pod \"kube-auth-proxy-7656b89f9d-ccn4t\" (UID: \"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5\") " pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:10.598518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.598470 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-tmp\") pod \"kube-auth-proxy-7656b89f9d-ccn4t\" (UID: \"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5\") " pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:10.699164 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.699123 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-tmp\") pod \"kube-auth-proxy-7656b89f9d-ccn4t\" (UID: \"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5\") " pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:10.699368 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.699172 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xczt\" (UniqueName: \"kubernetes.io/projected/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-kube-api-access-7xczt\") pod \"kube-auth-proxy-7656b89f9d-ccn4t\" (UID: \"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5\") " pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:10.699368 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.699200 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51107ee4-db1f-4dfc-b67c-f43269462379-cert\") pod \"odh-model-controller-7998c67dc4-szvmv\" (UID: \"51107ee4-db1f-4dfc-b67c-f43269462379\") " pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:10.699368 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:08:10.699291 2571 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 02 19:08:10.699368 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.699313 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p957\" (UniqueName: \"kubernetes.io/projected/51107ee4-db1f-4dfc-b67c-f43269462379-kube-api-access-8p957\") pod \"odh-model-controller-7998c67dc4-szvmv\" (UID: \"51107ee4-db1f-4dfc-b67c-f43269462379\") " pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:10.699368 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:08:10.699356 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51107ee4-db1f-4dfc-b67c-f43269462379-cert podName:51107ee4-db1f-4dfc-b67c-f43269462379 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:11.199339367 +0000 UTC m=+328.490676262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/51107ee4-db1f-4dfc-b67c-f43269462379-cert") pod "odh-model-controller-7998c67dc4-szvmv" (UID: "51107ee4-db1f-4dfc-b67c-f43269462379") : secret "odh-model-controller-webhook-cert" not found Jul 02 19:08:10.699654 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.699389 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-tls-certs\") pod \"kube-auth-proxy-7656b89f9d-ccn4t\" (UID: \"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5\") " pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:10.699654 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:08:10.699485 2571 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 02 19:08:10.699654 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:08:10.699532 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-tls-certs podName:7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:11.199517423 +0000 UTC m=+328.490854302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-tls-certs") pod "kube-auth-proxy-7656b89f9d-ccn4t" (UID: "7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5") : secret "kube-auth-proxy-tls" not found Jul 02 19:08:10.701352 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.701335 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-tmp\") pod \"kube-auth-proxy-7656b89f9d-ccn4t\" (UID: \"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5\") " pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:10.721547 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.721518 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p957\" (UniqueName: \"kubernetes.io/projected/51107ee4-db1f-4dfc-b67c-f43269462379-kube-api-access-8p957\") pod \"odh-model-controller-7998c67dc4-szvmv\" (UID: \"51107ee4-db1f-4dfc-b67c-f43269462379\") " pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:10.721881 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:10.721856 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xczt\" (UniqueName: \"kubernetes.io/projected/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-kube-api-access-7xczt\") pod \"kube-auth-proxy-7656b89f9d-ccn4t\" (UID: \"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5\") " pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:11.204431 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.204389 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-tls-certs\") pod \"kube-auth-proxy-7656b89f9d-ccn4t\" (UID: \"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5\") " pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:11.204606 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.204563 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51107ee4-db1f-4dfc-b67c-f43269462379-cert\") pod \"odh-model-controller-7998c67dc4-szvmv\" (UID: \"51107ee4-db1f-4dfc-b67c-f43269462379\") " pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:11.204698 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:08:11.204684 2571 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 02 19:08:11.204758 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:08:11.204749 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51107ee4-db1f-4dfc-b67c-f43269462379-cert podName:51107ee4-db1f-4dfc-b67c-f43269462379 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:12.204735024 +0000 UTC m=+329.496071902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/51107ee4-db1f-4dfc-b67c-f43269462379-cert") pod "odh-model-controller-7998c67dc4-szvmv" (UID: "51107ee4-db1f-4dfc-b67c-f43269462379") : secret "odh-model-controller-webhook-cert" not found Jul 02 19:08:11.207014 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.206990 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5-tls-certs\") pod \"kube-auth-proxy-7656b89f9d-ccn4t\" (UID: \"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5\") " pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:11.307327 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.307289 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9jv86"] Jul 02 19:08:11.312366 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.312344 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" Jul 02 19:08:11.314979 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.314958 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 02 19:08:11.328438 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.328408 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9jv86"] Jul 02 19:08:11.404992 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.404956 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" Jul 02 19:08:11.406265 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.406242 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83fa00f0-2ff4-47c3-9af9-f1deb313a786-tls-certs\") pod \"model-serving-api-54d89bd5fc-9jv86\" (UID: \"83fa00f0-2ff4-47c3-9af9-f1deb313a786\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" Jul 02 19:08:11.406342 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.406301 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh8n5\" (UniqueName: \"kubernetes.io/projected/83fa00f0-2ff4-47c3-9af9-f1deb313a786-kube-api-access-mh8n5\") pod \"model-serving-api-54d89bd5fc-9jv86\" (UID: \"83fa00f0-2ff4-47c3-9af9-f1deb313a786\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" Jul 02 19:08:11.507695 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.507643 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83fa00f0-2ff4-47c3-9af9-f1deb313a786-tls-certs\") pod \"model-serving-api-54d89bd5fc-9jv86\" (UID: \"83fa00f0-2ff4-47c3-9af9-f1deb313a786\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" Jul 02 19:08:11.508150 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.507760 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh8n5\" (UniqueName: \"kubernetes.io/projected/83fa00f0-2ff4-47c3-9af9-f1deb313a786-kube-api-access-mh8n5\") pod \"model-serving-api-54d89bd5fc-9jv86\" (UID: \"83fa00f0-2ff4-47c3-9af9-f1deb313a786\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" Jul 02 19:08:11.508150 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:08:11.507768 2571 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 02 19:08:11.508150 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:08:11.507844 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83fa00f0-2ff4-47c3-9af9-f1deb313a786-tls-certs podName:83fa00f0-2ff4-47c3-9af9-f1deb313a786 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:12.007824868 +0000 UTC m=+329.299161764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/83fa00f0-2ff4-47c3-9af9-f1deb313a786-tls-certs") pod "model-serving-api-54d89bd5fc-9jv86" (UID: "83fa00f0-2ff4-47c3-9af9-f1deb313a786") : secret "model-serving-api-tls" not found Jul 02 19:08:11.524676 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.524648 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh8n5\" (UniqueName: \"kubernetes.io/projected/83fa00f0-2ff4-47c3-9af9-f1deb313a786-kube-api-access-mh8n5\") pod \"model-serving-api-54d89bd5fc-9jv86\" (UID: \"83fa00f0-2ff4-47c3-9af9-f1deb313a786\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" Jul 02 19:08:11.558004 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.557910 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t"] Jul 02 19:08:11.560834 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:08:11.560803 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7657fe49_4f8a_4f02_a0b7_5bc7ced7c1c5.slice/crio-d2b792ffb0f65fedfe4f5bd71b87188068e7325b71ca57bb876da2fc55c26778 WatchSource:0}: Error finding container d2b792ffb0f65fedfe4f5bd71b87188068e7325b71ca57bb876da2fc55c26778: Status 404 returned error can't find the container with id d2b792ffb0f65fedfe4f5bd71b87188068e7325b71ca57bb876da2fc55c26778 Jul 02 19:08:11.587035 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:11.586983 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" event={"ID":"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5","Type":"ContainerStarted","Data":"d2b792ffb0f65fedfe4f5bd71b87188068e7325b71ca57bb876da2fc55c26778"} Jul 02 19:08:12.012212 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.012179 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83fa00f0-2ff4-47c3-9af9-f1deb313a786-tls-certs\") pod \"model-serving-api-54d89bd5fc-9jv86\" (UID: \"83fa00f0-2ff4-47c3-9af9-f1deb313a786\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" Jul 02 19:08:12.014566 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.014544 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83fa00f0-2ff4-47c3-9af9-f1deb313a786-tls-certs\") pod \"model-serving-api-54d89bd5fc-9jv86\" (UID: \"83fa00f0-2ff4-47c3-9af9-f1deb313a786\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" Jul 02 19:08:12.213519 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.213482 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51107ee4-db1f-4dfc-b67c-f43269462379-cert\") pod \"odh-model-controller-7998c67dc4-szvmv\" (UID: \"51107ee4-db1f-4dfc-b67c-f43269462379\") " pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:12.215887 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.215854 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51107ee4-db1f-4dfc-b67c-f43269462379-cert\") pod \"odh-model-controller-7998c67dc4-szvmv\" (UID: \"51107ee4-db1f-4dfc-b67c-f43269462379\") " pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:12.222758 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.222731 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" Jul 02 19:08:12.310458 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.310426 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:12.376012 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.375978 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9jv86"] Jul 02 19:08:12.377683 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:08:12.377632 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83fa00f0_2ff4_47c3_9af9_f1deb313a786.slice/crio-509d2f005d5589a7b3c362947130ec0e7855007faf650c2db4bdab1f1156822c WatchSource:0}: Error finding container 509d2f005d5589a7b3c362947130ec0e7855007faf650c2db4bdab1f1156822c: Status 404 returned error can't find the container with id 509d2f005d5589a7b3c362947130ec0e7855007faf650c2db4bdab1f1156822c Jul 02 19:08:12.458380 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.458201 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-szvmv"] Jul 02 19:08:12.461551 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:08:12.461519 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51107ee4_db1f_4dfc_b67c_f43269462379.slice/crio-59305e997b0a15ef513a82341bf92f9501c2cb8e6ac7aef3de9263e0bdda790e WatchSource:0}: Error finding container 59305e997b0a15ef513a82341bf92f9501c2cb8e6ac7aef3de9263e0bdda790e: Status 404 returned error can't find the container with id 59305e997b0a15ef513a82341bf92f9501c2cb8e6ac7aef3de9263e0bdda790e Jul 02 19:08:12.594832 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.594735 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" event={"ID":"51107ee4-db1f-4dfc-b67c-f43269462379","Type":"ContainerStarted","Data":"59305e997b0a15ef513a82341bf92f9501c2cb8e6ac7aef3de9263e0bdda790e"} Jul 02 19:08:12.596260 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.596232 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" event={"ID":"83fa00f0-2ff4-47c3-9af9-f1deb313a786","Type":"ContainerStarted","Data":"509d2f005d5589a7b3c362947130ec0e7855007faf650c2db4bdab1f1156822c"} Jul 02 19:08:12.688007 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.687970 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b648r"] Jul 02 19:08:12.692559 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.692539 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.699726 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.699702 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 19:08:12.699857 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.699743 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 02 19:08:12.700123 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.700104 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 02 19:08:12.700123 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.700118 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 02 19:08:12.700650 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.700624 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-s62hh\"" Jul 02 19:08:12.713679 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.713657 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b648r"] Jul 02 19:08:12.819874 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.819845 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.819874 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.819874 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02221135-4034-470c-a6e1-344b7268db01-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.820136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.819904 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-896pd\" (UniqueName: \"kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-kube-api-access-896pd\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.820136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.819971 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-cacerts\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.820136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.820011 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02221135-4034-470c-a6e1-344b7268db01-local-certs\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.820136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.820039 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-istio-token\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.820136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.820121 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.921442 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.921362 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.921442 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.921431 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.921703 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.921451 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02221135-4034-470c-a6e1-344b7268db01-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.921703 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.921474 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-896pd\" (UniqueName: \"kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-kube-api-access-896pd\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.921703 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.921493 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-cacerts\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.921703 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.921516 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02221135-4034-470c-a6e1-344b7268db01-local-certs\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.921703 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.921675 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-istio-token\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.922216 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.922192 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02221135-4034-470c-a6e1-344b7268db01-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.923872 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.923841 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02221135-4034-470c-a6e1-344b7268db01-local-certs\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.923965 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.923874 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-cacerts\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.923965 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.923955 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.924106 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.923995 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.930636 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.930608 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-896pd\" (UniqueName: \"kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-kube-api-access-896pd\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:12.931270 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:12.931253 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-istio-token\") pod \"istiod-openshift-gateway-7574459656-b648r\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:13.002549 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:13.002507 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:13.195745 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:13.195712 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b648r"] Jul 02 19:08:13.606180 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:13.606140 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" event={"ID":"02221135-4034-470c-a6e1-344b7268db01","Type":"ContainerStarted","Data":"a47c2caee94ea473e7bde41a981009e472f3c0a32ca3ca19847696af1ec9bf01"} Jul 02 19:08:15.620002 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:15.619967 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" event={"ID":"83fa00f0-2ff4-47c3-9af9-f1deb313a786","Type":"ContainerStarted","Data":"435df26bbac1f9e54dba922ec36d4bc281f1b3673e17b3ea31b5cd616f63e187"} Jul 02 19:08:15.620506 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:15.620151 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" Jul 02 19:08:15.639356 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:15.639293 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" podStartSLOduration=1.624849931 podStartE2EDuration="4.639269178s" totalImagesPullingTime="3.014419247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:15.638120549 +0000 UTC m=+332.929457450" watchObservedRunningTime="2026-07-02 19:08:15.639269178 +0000 UTC m=+332.930606069" Jul 02 19:08:17.632428 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:17.632394 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" event={"ID":"7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5","Type":"ContainerStarted","Data":"611d7c772892d11bbee5df8ce3967946ab444b4c94e082980ea5b41bfd26eb6b"} Jul 02 19:08:17.633825 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:17.633799 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" event={"ID":"51107ee4-db1f-4dfc-b67c-f43269462379","Type":"ContainerStarted","Data":"7e33d870cf5dccff6dc6ada5f4c8b0759ad036f2da0940aae25a0aed7cc8ea2a"} Jul 02 19:08:17.633943 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:17.633927 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:17.652699 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:17.652640 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7656b89f9d-ccn4t" podStartSLOduration=2.421070241 podStartE2EDuration="7.652622559s" totalImagesPullingTime="5.231552318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:17.650499325 +0000 UTC m=+334.941836226" watchObservedRunningTime="2026-07-02 19:08:17.652622559 +0000 UTC m=+334.943959459" Jul 02 19:08:17.671999 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:17.671945 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" podStartSLOduration=3.341021962 podStartE2EDuration="7.671928695s" totalImagesPullingTime="4.330906733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:17.669324039 +0000 UTC m=+334.960660939" watchObservedRunningTime="2026-07-02 19:08:17.671928695 +0000 UTC m=+334.963265600" Jul 02 19:08:21.457765 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.457737 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-49bv6"] Jul 02 19:08:21.461215 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.461194 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" Jul 02 19:08:21.464571 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.464523 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ng4kq\"" Jul 02 19:08:21.464698 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.464622 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 02 19:08:21.470039 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.470017 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m"] Jul 02 19:08:21.473273 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.473252 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" Jul 02 19:08:21.475872 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.475854 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 02 19:08:21.476922 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.476902 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jwnjn\"" Jul 02 19:08:21.481598 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.481575 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-49bv6"] Jul 02 19:08:21.485796 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.485773 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m"] Jul 02 19:08:21.608728 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.608690 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cwbs\" (UniqueName: \"kubernetes.io/projected/ad9662ae-db56-40a3-95dd-f1768572d712-kube-api-access-8cwbs\") pod \"kserve-controller-manager-d6f4b65b8-49bv6\" (UID: \"ad9662ae-db56-40a3-95dd-f1768572d712\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" Jul 02 19:08:21.608728 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.608727 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad9662ae-db56-40a3-95dd-f1768572d712-cert\") pod \"kserve-controller-manager-d6f4b65b8-49bv6\" (UID: \"ad9662ae-db56-40a3-95dd-f1768572d712\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" Jul 02 19:08:21.608942 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.608747 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/128fa7ca-3973-4884-830e-5b88f8bf187e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jvv9m\" (UID: \"128fa7ca-3973-4884-830e-5b88f8bf187e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" Jul 02 19:08:21.608942 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.608842 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnxph\" (UniqueName: \"kubernetes.io/projected/128fa7ca-3973-4884-830e-5b88f8bf187e-kube-api-access-cnxph\") pod \"llmisvc-controller-manager-777c6bd7d5-jvv9m\" (UID: \"128fa7ca-3973-4884-830e-5b88f8bf187e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" Jul 02 19:08:21.710417 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.710324 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cwbs\" (UniqueName: \"kubernetes.io/projected/ad9662ae-db56-40a3-95dd-f1768572d712-kube-api-access-8cwbs\") pod \"kserve-controller-manager-d6f4b65b8-49bv6\" (UID: \"ad9662ae-db56-40a3-95dd-f1768572d712\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" Jul 02 19:08:21.710417 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.710376 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad9662ae-db56-40a3-95dd-f1768572d712-cert\") pod \"kserve-controller-manager-d6f4b65b8-49bv6\" (UID: \"ad9662ae-db56-40a3-95dd-f1768572d712\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" Jul 02 19:08:21.710417 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.710401 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/128fa7ca-3973-4884-830e-5b88f8bf187e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jvv9m\" (UID: \"128fa7ca-3973-4884-830e-5b88f8bf187e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" Jul 02 19:08:21.710635 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.710449 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnxph\" (UniqueName: \"kubernetes.io/projected/128fa7ca-3973-4884-830e-5b88f8bf187e-kube-api-access-cnxph\") pod \"llmisvc-controller-manager-777c6bd7d5-jvv9m\" (UID: \"128fa7ca-3973-4884-830e-5b88f8bf187e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" Jul 02 19:08:21.710635 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:08:21.710550 2571 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 02 19:08:21.710635 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:08:21.710633 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad9662ae-db56-40a3-95dd-f1768572d712-cert podName:ad9662ae-db56-40a3-95dd-f1768572d712 nodeName:}" failed. No retries permitted until 2026-07-02 19:08:22.210614809 +0000 UTC m=+339.501951690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ad9662ae-db56-40a3-95dd-f1768572d712-cert") pod "kserve-controller-manager-d6f4b65b8-49bv6" (UID: "ad9662ae-db56-40a3-95dd-f1768572d712") : secret "kserve-webhook-server-cert" not found Jul 02 19:08:21.712863 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.712845 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/128fa7ca-3973-4884-830e-5b88f8bf187e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jvv9m\" (UID: \"128fa7ca-3973-4884-830e-5b88f8bf187e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" Jul 02 19:08:21.721114 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.721091 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnxph\" (UniqueName: \"kubernetes.io/projected/128fa7ca-3973-4884-830e-5b88f8bf187e-kube-api-access-cnxph\") pod \"llmisvc-controller-manager-777c6bd7d5-jvv9m\" (UID: \"128fa7ca-3973-4884-830e-5b88f8bf187e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" Jul 02 19:08:21.724998 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.724976 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cwbs\" (UniqueName: \"kubernetes.io/projected/ad9662ae-db56-40a3-95dd-f1768572d712-kube-api-access-8cwbs\") pod \"kserve-controller-manager-d6f4b65b8-49bv6\" (UID: \"ad9662ae-db56-40a3-95dd-f1768572d712\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" Jul 02 19:08:21.784050 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.784012 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" Jul 02 19:08:21.969853 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:21.969749 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m"] Jul 02 19:08:21.985765 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:08:21.985499 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod128fa7ca_3973_4884_830e_5b88f8bf187e.slice/crio-1de8df6a323555155b90f1193eb68856b91400a38e7f1400e1820c4caa7a028c WatchSource:0}: Error finding container 1de8df6a323555155b90f1193eb68856b91400a38e7f1400e1820c4caa7a028c: Status 404 returned error can't find the container with id 1de8df6a323555155b90f1193eb68856b91400a38e7f1400e1820c4caa7a028c Jul 02 19:08:22.216817 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:22.216775 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad9662ae-db56-40a3-95dd-f1768572d712-cert\") pod \"kserve-controller-manager-d6f4b65b8-49bv6\" (UID: \"ad9662ae-db56-40a3-95dd-f1768572d712\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" Jul 02 19:08:22.219658 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:22.219626 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad9662ae-db56-40a3-95dd-f1768572d712-cert\") pod \"kserve-controller-manager-d6f4b65b8-49bv6\" (UID: \"ad9662ae-db56-40a3-95dd-f1768572d712\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" Jul 02 19:08:22.371523 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:22.371487 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" Jul 02 19:08:22.538371 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:22.538335 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-49bv6"] Jul 02 19:08:22.541176 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:08:22.541143 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9662ae_db56_40a3_95dd_f1768572d712.slice/crio-9b9e7547efe07e82fa8e8e4a2c7544e5e8111abc12fff1903cb091149a37fa13 WatchSource:0}: Error finding container 9b9e7547efe07e82fa8e8e4a2c7544e5e8111abc12fff1903cb091149a37fa13: Status 404 returned error can't find the container with id 9b9e7547efe07e82fa8e8e4a2c7544e5e8111abc12fff1903cb091149a37fa13 Jul 02 19:08:22.660336 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:22.660229 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" event={"ID":"ad9662ae-db56-40a3-95dd-f1768572d712","Type":"ContainerStarted","Data":"9b9e7547efe07e82fa8e8e4a2c7544e5e8111abc12fff1903cb091149a37fa13"} Jul 02 19:08:22.661877 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:22.661845 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" event={"ID":"128fa7ca-3973-4884-830e-5b88f8bf187e","Type":"ContainerStarted","Data":"1de8df6a323555155b90f1193eb68856b91400a38e7f1400e1820c4caa7a028c"} Jul 02 19:08:24.264259 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:24.264214 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:08:24.264604 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:24.264286 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:08:24.678670 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:24.678632 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" event={"ID":"02221135-4034-470c-a6e1-344b7268db01","Type":"ContainerStarted","Data":"2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903"} Jul 02 19:08:24.678819 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:24.678740 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:24.725676 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:24.725619 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" podStartSLOduration=1.663548043 podStartE2EDuration="12.725599926s" totalImagesPullingTime="11.062051883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:24.720885829 +0000 UTC m=+342.012222732" watchObservedRunningTime="2026-07-02 19:08:24.725599926 +0000 UTC m=+342.016936826" Jul 02 19:08:25.210542 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:25.210499 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b648r"] Jul 02 19:08:25.687236 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:25.687208 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:26.630504 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:26.630479 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9jv86" Jul 02 19:08:26.693025 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:26.692975 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" event={"ID":"128fa7ca-3973-4884-830e-5b88f8bf187e","Type":"ContainerStarted","Data":"d1f390207fe4ef2f98a1dc53e38cc867c14c869833422ee4dfc6dc5148861e87"} Jul 02 19:08:26.693507 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:26.693139 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" Jul 02 19:08:26.693507 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:26.693314 2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" podUID="02221135-4034-470c-a6e1-344b7268db01" containerName="discovery" containerID="cri-o://2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903" gracePeriod=30 Jul 02 19:08:26.716354 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:26.716295 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" podStartSLOduration=1.997773193 podStartE2EDuration="5.716272948s" totalImagesPullingTime="3.718499755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:26.714168473 +0000 UTC m=+344.005505375" watchObservedRunningTime="2026-07-02 19:08:26.716272948 +0000 UTC m=+344.007609877" Jul 02 19:08:27.424217 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.424195 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:27.571136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.571037 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-csr-dns-cert\") pod \"02221135-4034-470c-a6e1-344b7268db01\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " Jul 02 19:08:27.571136 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.571113 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02221135-4034-470c-a6e1-344b7268db01-istio-csr-ca-configmap\") pod \"02221135-4034-470c-a6e1-344b7268db01\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " Jul 02 19:08:27.571314 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.571224 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02221135-4034-470c-a6e1-344b7268db01-local-certs\") pod \"02221135-4034-470c-a6e1-344b7268db01\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " Jul 02 19:08:27.571314 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.571293 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-cacerts\") pod \"02221135-4034-470c-a6e1-344b7268db01\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " Jul 02 19:08:27.571378 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.571327 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-896pd\" (UniqueName: \"kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-kube-api-access-896pd\") pod \"02221135-4034-470c-a6e1-344b7268db01\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " Jul 02 19:08:27.571378 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.571361 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-istio-token\") pod \"02221135-4034-470c-a6e1-344b7268db01\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " Jul 02 19:08:27.571486 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.571393 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-kubeconfig\") pod \"02221135-4034-470c-a6e1-344b7268db01\" (UID: \"02221135-4034-470c-a6e1-344b7268db01\") " Jul 02 19:08:27.571486 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.571450 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02221135-4034-470c-a6e1-344b7268db01-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "02221135-4034-470c-a6e1-344b7268db01" (UID: "02221135-4034-470c-a6e1-344b7268db01"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:08:27.571752 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.571728 2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02221135-4034-470c-a6e1-344b7268db01-istio-csr-ca-configmap\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:08:27.573550 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.573516 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02221135-4034-470c-a6e1-344b7268db01-local-certs" (OuterVolumeSpecName: "local-certs") pod "02221135-4034-470c-a6e1-344b7268db01" (UID: "02221135-4034-470c-a6e1-344b7268db01"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:08:27.573550 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.573540 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-kube-api-access-896pd" (OuterVolumeSpecName: "kube-api-access-896pd") pod "02221135-4034-470c-a6e1-344b7268db01" (UID: "02221135-4034-470c-a6e1-344b7268db01"). InnerVolumeSpecName "kube-api-access-896pd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:08:27.573687 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.573639 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-istio-token" (OuterVolumeSpecName: "istio-token") pod "02221135-4034-470c-a6e1-344b7268db01" (UID: "02221135-4034-470c-a6e1-344b7268db01"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:08:27.573687 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.573662 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "02221135-4034-470c-a6e1-344b7268db01" (UID: "02221135-4034-470c-a6e1-344b7268db01"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:27.573857 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.573839 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-cacerts" (OuterVolumeSpecName: "cacerts") pod "02221135-4034-470c-a6e1-344b7268db01" (UID: "02221135-4034-470c-a6e1-344b7268db01"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:27.573912 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.573853 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "02221135-4034-470c-a6e1-344b7268db01" (UID: "02221135-4034-470c-a6e1-344b7268db01"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:27.673157 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.673118 2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-csr-dns-cert\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:08:27.673157 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.673148 2571 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02221135-4034-470c-a6e1-344b7268db01-local-certs\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:08:27.673157 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.673160 2571 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-cacerts\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:08:27.673452 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.673171 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-896pd\" (UniqueName: \"kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-kube-api-access-896pd\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:08:27.673452 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.673181 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02221135-4034-470c-a6e1-344b7268db01-istio-token\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:08:27.673452 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.673189 2571 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02221135-4034-470c-a6e1-344b7268db01-istio-kubeconfig\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:08:27.698705 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.698671 2571 generic.go:356] "Generic (PLEG): container finished" podID="02221135-4034-470c-a6e1-344b7268db01" containerID="2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903" exitCode=0 Jul 02 19:08:27.699146 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.698721 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" Jul 02 19:08:27.699146 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.698757 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" event={"ID":"02221135-4034-470c-a6e1-344b7268db01","Type":"ContainerDied","Data":"2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903"} Jul 02 19:08:27.699146 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.698813 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b648r" event={"ID":"02221135-4034-470c-a6e1-344b7268db01","Type":"ContainerDied","Data":"a47c2caee94ea473e7bde41a981009e472f3c0a32ca3ca19847696af1ec9bf01"} Jul 02 19:08:27.699146 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.698843 2571 scope.go:117] "RemoveContainer" containerID="2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903" Jul 02 19:08:27.700738 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.700709 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" event={"ID":"ad9662ae-db56-40a3-95dd-f1768572d712","Type":"ContainerStarted","Data":"1575847de09bc0e00d5ffe52dddcb2e6c7e13610f6e713035af98f375eb46899"} Jul 02 19:08:27.700854 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.700834 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" Jul 02 19:08:27.708995 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.708977 2571 scope.go:117] "RemoveContainer" containerID="2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903" Jul 02 19:08:27.709384 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:08:27.709363 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903\": container with ID starting with 2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903 not found: ID does not exist" containerID="2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903" Jul 02 19:08:27.709458 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.709391 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903"} err="failed to get container status \"2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903\": rpc error: code = NotFound desc = could not find container \"2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903\": container with ID starting with 2003c778deccbd1a40184dcfcdb61799fadf535426d0cc2ccec6c90e8084e903 not found: ID does not exist" Jul 02 19:08:27.718686 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.718641 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" podStartSLOduration=1.930177562 podStartE2EDuration="6.718626717s" totalImagesPullingTime="4.788449155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:27.717448746 +0000 UTC m=+345.008785647" watchObservedRunningTime="2026-07-02 19:08:27.718626717 +0000 UTC m=+345.009963755" Jul 02 19:08:27.733408 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.733377 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b648r"] Jul 02 19:08:27.736339 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:27.736314 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b648r"] Jul 02 19:08:28.642118 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:28.642093 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-szvmv" Jul 02 19:08:29.238916 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:29.238885 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02221135-4034-470c-a6e1-344b7268db01" path="/var/lib/kubelet/pods/02221135-4034-470c-a6e1-344b7268db01/volumes" Jul 02 19:08:57.702526 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:57.702493 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jvv9m" Jul 02 19:08:58.712601 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:08:58.712564 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-49bv6" Jul 02 19:09:08.395984 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.395944 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w49dr"] Jul 02 19:09:08.396467 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.396353 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02221135-4034-470c-a6e1-344b7268db01" containerName="discovery" Jul 02 19:09:08.396467 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.396367 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="02221135-4034-470c-a6e1-344b7268db01" containerName="discovery" Jul 02 19:09:08.396467 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.396421 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="02221135-4034-470c-a6e1-344b7268db01" containerName="discovery" Jul 02 19:09:08.398395 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.398380 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w49dr" Jul 02 19:09:08.401157 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.401133 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 19:09:08.401343 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.401326 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 19:09:08.401998 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.401983 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-nwjvk\"" Jul 02 19:09:08.412125 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.412100 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w49dr"] Jul 02 19:09:08.529416 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.529365 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5db4k\" (UniqueName: \"kubernetes.io/projected/f73900ff-9e2f-4e63-bf79-74fd79c2747f-kube-api-access-5db4k\") pod \"kuadrant-operator-catalog-w49dr\" (UID: \"f73900ff-9e2f-4e63-bf79-74fd79c2747f\") " pod="kuadrant-system/kuadrant-operator-catalog-w49dr" Jul 02 19:09:08.630914 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.630876 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5db4k\" (UniqueName: \"kubernetes.io/projected/f73900ff-9e2f-4e63-bf79-74fd79c2747f-kube-api-access-5db4k\") pod \"kuadrant-operator-catalog-w49dr\" (UID: \"f73900ff-9e2f-4e63-bf79-74fd79c2747f\") " pod="kuadrant-system/kuadrant-operator-catalog-w49dr" Jul 02 19:09:08.639696 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.639667 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5db4k\" (UniqueName: \"kubernetes.io/projected/f73900ff-9e2f-4e63-bf79-74fd79c2747f-kube-api-access-5db4k\") pod \"kuadrant-operator-catalog-w49dr\" (UID: \"f73900ff-9e2f-4e63-bf79-74fd79c2747f\") " pod="kuadrant-system/kuadrant-operator-catalog-w49dr" Jul 02 19:09:08.708433 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.708344 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w49dr" Jul 02 19:09:08.753811 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.753777 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w49dr"] Jul 02 19:09:08.862564 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.862539 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w49dr"] Jul 02 19:09:08.864162 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:09:08.864133 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73900ff_9e2f_4e63_bf79_74fd79c2747f.slice/crio-a5dd9c732ab6bd05f8419a9ab97e19c058fafc429865611c06c5dbbc7adddec6 WatchSource:0}: Error finding container a5dd9c732ab6bd05f8419a9ab97e19c058fafc429865611c06c5dbbc7adddec6: Status 404 returned error can't find the container with id a5dd9c732ab6bd05f8419a9ab97e19c058fafc429865611c06c5dbbc7adddec6 Jul 02 19:09:08.891777 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.891744 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w49dr" event={"ID":"f73900ff-9e2f-4e63-bf79-74fd79c2747f","Type":"ContainerStarted","Data":"a5dd9c732ab6bd05f8419a9ab97e19c058fafc429865611c06c5dbbc7adddec6"} Jul 02 19:09:08.964248 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.964172 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6f828"] Jul 02 19:09:08.967000 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.966985 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6f828" Jul 02 19:09:08.976447 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:08.976418 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6f828"] Jul 02 19:09:09.136469 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:09.136437 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgp57\" (UniqueName: \"kubernetes.io/projected/1c6b86a0-461b-4bec-a4ae-b63d652580b9-kube-api-access-rgp57\") pod \"kuadrant-operator-catalog-6f828\" (UID: \"1c6b86a0-461b-4bec-a4ae-b63d652580b9\") " pod="kuadrant-system/kuadrant-operator-catalog-6f828" Jul 02 19:09:09.237188 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:09.237103 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgp57\" (UniqueName: \"kubernetes.io/projected/1c6b86a0-461b-4bec-a4ae-b63d652580b9-kube-api-access-rgp57\") pod \"kuadrant-operator-catalog-6f828\" (UID: \"1c6b86a0-461b-4bec-a4ae-b63d652580b9\") " pod="kuadrant-system/kuadrant-operator-catalog-6f828" Jul 02 19:09:09.246173 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:09.246142 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgp57\" (UniqueName: \"kubernetes.io/projected/1c6b86a0-461b-4bec-a4ae-b63d652580b9-kube-api-access-rgp57\") pod \"kuadrant-operator-catalog-6f828\" (UID: \"1c6b86a0-461b-4bec-a4ae-b63d652580b9\") " pod="kuadrant-system/kuadrant-operator-catalog-6f828" Jul 02 19:09:09.277245 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:09.277208 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6f828" Jul 02 19:09:09.422126 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:09.422103 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6f828"] Jul 02 19:09:09.424836 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:09:09.424806 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6b86a0_461b_4bec_a4ae_b63d652580b9.slice/crio-e115d23ab6e8639a57e058284fd6b1a9d3594dcc7a84e21d401af440d659274e WatchSource:0}: Error finding container e115d23ab6e8639a57e058284fd6b1a9d3594dcc7a84e21d401af440d659274e: Status 404 returned error can't find the container with id e115d23ab6e8639a57e058284fd6b1a9d3594dcc7a84e21d401af440d659274e Jul 02 19:09:09.901501 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:09.901459 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6f828" event={"ID":"1c6b86a0-461b-4bec-a4ae-b63d652580b9","Type":"ContainerStarted","Data":"e115d23ab6e8639a57e058284fd6b1a9d3594dcc7a84e21d401af440d659274e"} Jul 02 19:09:11.911981 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:11.911943 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w49dr" event={"ID":"f73900ff-9e2f-4e63-bf79-74fd79c2747f","Type":"ContainerStarted","Data":"2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4"} Jul 02 19:09:11.912506 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:11.911980 2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-w49dr" podUID="f73900ff-9e2f-4e63-bf79-74fd79c2747f" containerName="registry-server" containerID="cri-o://2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4" gracePeriod=2 Jul 02 19:09:11.913408 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:11.913381 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6f828" event={"ID":"1c6b86a0-461b-4bec-a4ae-b63d652580b9","Type":"ContainerStarted","Data":"637033566ccb0bbb5a4848b6abd602b1e56f67c41ec05b87afb5a6c1937c5707"} Jul 02 19:09:11.937757 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:11.937701 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-w49dr" podStartSLOduration=1.461871452 podStartE2EDuration="3.937683134s" totalImagesPullingTime="2.475811682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:11.935669683 +0000 UTC m=+389.227006585" watchObservedRunningTime="2026-07-02 19:09:11.937683134 +0000 UTC m=+389.229020033" Jul 02 19:09:11.954799 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:11.954748 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6f828" podStartSLOduration=2.043095365 podStartE2EDuration="3.95473103s" totalImagesPullingTime="1.911635665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:11.954272801 +0000 UTC m=+389.245609703" watchObservedRunningTime="2026-07-02 19:09:11.95473103 +0000 UTC m=+389.246067977" Jul 02 19:09:12.162483 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.162421 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w49dr" Jul 02 19:09:12.264335 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.264304 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5db4k\" (UniqueName: \"kubernetes.io/projected/f73900ff-9e2f-4e63-bf79-74fd79c2747f-kube-api-access-5db4k\") pod \"f73900ff-9e2f-4e63-bf79-74fd79c2747f\" (UID: \"f73900ff-9e2f-4e63-bf79-74fd79c2747f\") " Jul 02 19:09:12.266410 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.266388 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f73900ff-9e2f-4e63-bf79-74fd79c2747f-kube-api-access-5db4k" (OuterVolumeSpecName: "kube-api-access-5db4k") pod "f73900ff-9e2f-4e63-bf79-74fd79c2747f" (UID: "f73900ff-9e2f-4e63-bf79-74fd79c2747f"). InnerVolumeSpecName "kube-api-access-5db4k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:09:12.365861 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.365834 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5db4k\" (UniqueName: \"kubernetes.io/projected/f73900ff-9e2f-4e63-bf79-74fd79c2747f-kube-api-access-5db4k\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:12.919830 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.919791 2571 generic.go:356] "Generic (PLEG): container finished" podID="f73900ff-9e2f-4e63-bf79-74fd79c2747f" containerID="2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4" exitCode=0 Jul 02 19:09:12.919830 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.919834 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w49dr" Jul 02 19:09:12.920354 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.919883 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w49dr" event={"ID":"f73900ff-9e2f-4e63-bf79-74fd79c2747f","Type":"ContainerDied","Data":"2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4"} Jul 02 19:09:12.920354 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.919972 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w49dr" event={"ID":"f73900ff-9e2f-4e63-bf79-74fd79c2747f","Type":"ContainerDied","Data":"a5dd9c732ab6bd05f8419a9ab97e19c058fafc429865611c06c5dbbc7adddec6"} Jul 02 19:09:12.920354 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.920001 2571 scope.go:117] "RemoveContainer" containerID="2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4" Jul 02 19:09:12.929847 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.929827 2571 scope.go:117] "RemoveContainer" containerID="2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4" Jul 02 19:09:12.930142 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:09:12.930122 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4\": container with ID starting with 2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4 not found: ID does not exist" containerID="2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4" Jul 02 19:09:12.930262 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.930152 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4"} err="failed to get container status \"2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4\": rpc error: code = NotFound desc = could not find container \"2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4\": container with ID starting with 2bd04b3e630e81865d42abeb58d858ccd438be151e41643fb9c62a6dc9adf3a4 not found: ID does not exist" Jul 02 19:09:12.942978 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.942953 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w49dr"] Jul 02 19:09:12.950899 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:12.950877 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w49dr"] Jul 02 19:09:13.238453 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:13.238371 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f73900ff-9e2f-4e63-bf79-74fd79c2747f" path="/var/lib/kubelet/pods/f73900ff-9e2f-4e63-bf79-74fd79c2747f/volumes" Jul 02 19:09:19.277562 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:19.277512 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6f828" Jul 02 19:09:19.278061 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:19.277584 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6f828" Jul 02 19:09:19.299876 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:19.299843 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6f828" Jul 02 19:09:19.977017 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:19.976992 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6f828" Jul 02 19:09:24.064411 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.064376 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm"] Jul 02 19:09:24.064782 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.064714 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73900ff-9e2f-4e63-bf79-74fd79c2747f" containerName="registry-server" Jul 02 19:09:24.064782 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.064725 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73900ff-9e2f-4e63-bf79-74fd79c2747f" containerName="registry-server" Jul 02 19:09:24.064782 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.064783 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f73900ff-9e2f-4e63-bf79-74fd79c2747f" containerName="registry-server" Jul 02 19:09:24.069401 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.069380 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:24.073350 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.073328 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-s9r5w\"" Jul 02 19:09:24.084718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.084692 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm"] Jul 02 19:09:24.166514 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.166476 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:24.166514 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.166513 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:24.166725 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.166620 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd8cw\" (UniqueName: \"kubernetes.io/projected/e6d8b20e-5192-482a-af1f-33e3ba49ec74-kube-api-access-dd8cw\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:24.267949 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.267901 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd8cw\" (UniqueName: \"kubernetes.io/projected/e6d8b20e-5192-482a-af1f-33e3ba49ec74-kube-api-access-dd8cw\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:24.268163 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.267965 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:24.268163 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.267993 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:24.268394 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.268375 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:24.268431 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.268402 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:24.279090 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.279039 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd8cw\" (UniqueName: \"kubernetes.io/projected/e6d8b20e-5192-482a-af1f-33e3ba49ec74-kube-api-access-dd8cw\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:24.378563 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.378479 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:24.414225 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.414194 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz"] Jul 02 19:09:24.419036 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.419009 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:24.431486 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.431454 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz"] Jul 02 19:09:24.469713 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.469678 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twg8q\" (UniqueName: \"kubernetes.io/projected/164cd298-6237-4212-aa0b-cb6004c464f2-kube-api-access-twg8q\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:24.469863 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.469727 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:24.469863 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.469763 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:24.523960 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.523928 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm"] Jul 02 19:09:24.527533 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:09:24.527498 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d8b20e_5192_482a_af1f_33e3ba49ec74.slice/crio-20519b99c82990b6148aa20f82eb3c1e6b8b1f596f9f75dde1f7cbe1fae99785 WatchSource:0}: Error finding container 20519b99c82990b6148aa20f82eb3c1e6b8b1f596f9f75dde1f7cbe1fae99785: Status 404 returned error can't find the container with id 20519b99c82990b6148aa20f82eb3c1e6b8b1f596f9f75dde1f7cbe1fae99785 Jul 02 19:09:24.570191 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.570167 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:24.570307 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.570198 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:24.570368 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.570304 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twg8q\" (UniqueName: \"kubernetes.io/projected/164cd298-6237-4212-aa0b-cb6004c464f2-kube-api-access-twg8q\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:24.570585 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.570565 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:24.570649 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.570590 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:24.579170 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.579147 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twg8q\" (UniqueName: \"kubernetes.io/projected/164cd298-6237-4212-aa0b-cb6004c464f2-kube-api-access-twg8q\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:24.730179 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.730145 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:24.879062 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.879030 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz"] Jul 02 19:09:24.882180 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.882118 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5"] Jul 02 19:09:24.882180 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:09:24.882159 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164cd298_6237_4212_aa0b_cb6004c464f2.slice/crio-c7fdd2b02f0778ab86495dfbca08f981db4a59546e4041147ad2539e43a2f5f9 WatchSource:0}: Error finding container c7fdd2b02f0778ab86495dfbca08f981db4a59546e4041147ad2539e43a2f5f9: Status 404 returned error can't find the container with id c7fdd2b02f0778ab86495dfbca08f981db4a59546e4041147ad2539e43a2f5f9 Jul 02 19:09:24.887372 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.887352 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:24.908143 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.908056 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5"] Jul 02 19:09:24.974027 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.973995 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:24.974218 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.974048 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49t4j\" (UniqueName: \"kubernetes.io/projected/a378be6c-22a8-419d-b887-2619dfb8024f-kube-api-access-49t4j\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:24.974218 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.974149 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:24.979705 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.979656 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" event={"ID":"164cd298-6237-4212-aa0b-cb6004c464f2","Type":"ContainerStarted","Data":"7196b9a5c2ecbc6b2c5e7fff7f8d6f0c8242217787c32420bbccc53240547d08"} Jul 02 19:09:24.979889 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.979717 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" event={"ID":"164cd298-6237-4212-aa0b-cb6004c464f2","Type":"ContainerStarted","Data":"c7fdd2b02f0778ab86495dfbca08f981db4a59546e4041147ad2539e43a2f5f9"} Jul 02 19:09:24.981454 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.981423 2571 generic.go:356] "Generic (PLEG): container finished" podID="e6d8b20e-5192-482a-af1f-33e3ba49ec74" containerID="00183f95190cc89f4a2756ec4a6ee1a10800c973403c44c3d1f9d19c93119a45" exitCode=0 Jul 02 19:09:24.981571 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.981503 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" event={"ID":"e6d8b20e-5192-482a-af1f-33e3ba49ec74","Type":"ContainerDied","Data":"00183f95190cc89f4a2756ec4a6ee1a10800c973403c44c3d1f9d19c93119a45"} Jul 02 19:09:24.981571 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:24.981542 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" event={"ID":"e6d8b20e-5192-482a-af1f-33e3ba49ec74","Type":"ContainerStarted","Data":"20519b99c82990b6148aa20f82eb3c1e6b8b1f596f9f75dde1f7cbe1fae99785"} Jul 02 19:09:25.076918 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.076875 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:25.077405 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.076976 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:25.077405 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.077033 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49t4j\" (UniqueName: \"kubernetes.io/projected/a378be6c-22a8-419d-b887-2619dfb8024f-kube-api-access-49t4j\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:25.077405 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.077380 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:25.077550 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.077440 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:25.091582 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.091554 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49t4j\" (UniqueName: \"kubernetes.io/projected/a378be6c-22a8-419d-b887-2619dfb8024f-kube-api-access-49t4j\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:25.196963 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.196863 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:25.261915 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.261883 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh"] Jul 02 19:09:25.266572 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.266549 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:25.278684 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.278657 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:25.278847 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.278718 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:25.278847 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.278744 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsqlx\" (UniqueName: \"kubernetes.io/projected/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-kube-api-access-qsqlx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:25.283308 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.283284 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh"] Jul 02 19:09:25.358168 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.358139 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5"] Jul 02 19:09:25.361370 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:09:25.361344 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda378be6c_22a8_419d_b887_2619dfb8024f.slice/crio-c706853e5bfadb253c814568a94dd24a8a4afa2047e0b8e861d3f78d714363fd WatchSource:0}: Error finding container c706853e5bfadb253c814568a94dd24a8a4afa2047e0b8e861d3f78d714363fd: Status 404 returned error can't find the container with id c706853e5bfadb253c814568a94dd24a8a4afa2047e0b8e861d3f78d714363fd Jul 02 19:09:25.380117 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.380046 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:25.380216 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.380142 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:25.380216 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.380169 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsqlx\" (UniqueName: \"kubernetes.io/projected/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-kube-api-access-qsqlx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:25.380454 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.380433 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:25.380454 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.380444 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:25.421910 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.421841 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsqlx\" (UniqueName: \"kubernetes.io/projected/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-kube-api-access-qsqlx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:25.575351 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.575318 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:25.724753 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.724668 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh"] Jul 02 19:09:25.727457 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:09:25.727432 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80e2a77b_0851_4bc5_8ed0_c6d49ec62683.slice/crio-f98f53d3258586640ea5d1dd2185d62643231981e89c87d6bb3ee1efabe559a8 WatchSource:0}: Error finding container f98f53d3258586640ea5d1dd2185d62643231981e89c87d6bb3ee1efabe559a8: Status 404 returned error can't find the container with id f98f53d3258586640ea5d1dd2185d62643231981e89c87d6bb3ee1efabe559a8 Jul 02 19:09:25.988186 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.988096 2571 generic.go:356] "Generic (PLEG): container finished" podID="164cd298-6237-4212-aa0b-cb6004c464f2" containerID="7196b9a5c2ecbc6b2c5e7fff7f8d6f0c8242217787c32420bbccc53240547d08" exitCode=0 Jul 02 19:09:25.988186 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.988142 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" event={"ID":"164cd298-6237-4212-aa0b-cb6004c464f2","Type":"ContainerDied","Data":"7196b9a5c2ecbc6b2c5e7fff7f8d6f0c8242217787c32420bbccc53240547d08"} Jul 02 19:09:25.989809 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.989781 2571 generic.go:356] "Generic (PLEG): container finished" podID="80e2a77b-0851-4bc5-8ed0-c6d49ec62683" containerID="0f5438d4a2e9372017123e2b5ed28a9046d451f70f3772fa06eb4d996bef34e5" exitCode=0 Jul 02 19:09:25.990182 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.989858 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" event={"ID":"80e2a77b-0851-4bc5-8ed0-c6d49ec62683","Type":"ContainerDied","Data":"0f5438d4a2e9372017123e2b5ed28a9046d451f70f3772fa06eb4d996bef34e5"} Jul 02 19:09:25.990182 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.989906 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" event={"ID":"80e2a77b-0851-4bc5-8ed0-c6d49ec62683","Type":"ContainerStarted","Data":"f98f53d3258586640ea5d1dd2185d62643231981e89c87d6bb3ee1efabe559a8"} Jul 02 19:09:25.991512 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.991486 2571 generic.go:356] "Generic (PLEG): container finished" podID="a378be6c-22a8-419d-b887-2619dfb8024f" containerID="1a54011e52ee5f367d56d08cfc38e26d60d9bab25dc11282af4e9003d95fe7c0" exitCode=0 Jul 02 19:09:25.991608 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.991570 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" event={"ID":"a378be6c-22a8-419d-b887-2619dfb8024f","Type":"ContainerDied","Data":"1a54011e52ee5f367d56d08cfc38e26d60d9bab25dc11282af4e9003d95fe7c0"} Jul 02 19:09:25.991663 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:25.991625 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" event={"ID":"a378be6c-22a8-419d-b887-2619dfb8024f","Type":"ContainerStarted","Data":"c706853e5bfadb253c814568a94dd24a8a4afa2047e0b8e861d3f78d714363fd"} Jul 02 19:09:26.998682 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:26.998652 2571 generic.go:356] "Generic (PLEG): container finished" podID="e6d8b20e-5192-482a-af1f-33e3ba49ec74" containerID="484a6f421215616988b7a0c105caebad0cf9f64c971db0e0ea4f228419281389" exitCode=0 Jul 02 19:09:26.999062 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:26.998732 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" event={"ID":"e6d8b20e-5192-482a-af1f-33e3ba49ec74","Type":"ContainerDied","Data":"484a6f421215616988b7a0c105caebad0cf9f64c971db0e0ea4f228419281389"} Jul 02 19:09:28.011526 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:28.011491 2571 generic.go:356] "Generic (PLEG): container finished" podID="80e2a77b-0851-4bc5-8ed0-c6d49ec62683" containerID="ddd6254d3eae7ab259aa5344ac3e2330a4fb319e7ea028c30671cb5097602c99" exitCode=0 Jul 02 19:09:28.011971 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:28.011562 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" event={"ID":"80e2a77b-0851-4bc5-8ed0-c6d49ec62683","Type":"ContainerDied","Data":"ddd6254d3eae7ab259aa5344ac3e2330a4fb319e7ea028c30671cb5097602c99"} Jul 02 19:09:28.013447 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:28.013419 2571 generic.go:356] "Generic (PLEG): container finished" podID="a378be6c-22a8-419d-b887-2619dfb8024f" containerID="9c5a78a156fd792638ce448b11e2ce2dc98aab5b2dcc0bd534304c7e74ef1710" exitCode=0 Jul 02 19:09:28.013515 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:28.013496 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" event={"ID":"a378be6c-22a8-419d-b887-2619dfb8024f","Type":"ContainerDied","Data":"9c5a78a156fd792638ce448b11e2ce2dc98aab5b2dcc0bd534304c7e74ef1710"} Jul 02 19:09:28.015482 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:28.015459 2571 generic.go:356] "Generic (PLEG): container finished" podID="164cd298-6237-4212-aa0b-cb6004c464f2" containerID="a191135f4947a1fe96ce21c78c446e94b3a97b98ef93f36cc0b8e364a1c2cca4" exitCode=0 Jul 02 19:09:28.015575 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:28.015540 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" event={"ID":"164cd298-6237-4212-aa0b-cb6004c464f2","Type":"ContainerDied","Data":"a191135f4947a1fe96ce21c78c446e94b3a97b98ef93f36cc0b8e364a1c2cca4"} Jul 02 19:09:28.017746 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:28.017727 2571 generic.go:356] "Generic (PLEG): container finished" podID="e6d8b20e-5192-482a-af1f-33e3ba49ec74" containerID="56cb846b537a577c29549c55c94ea74453b40ea72251f72066d74aa274e21c8d" exitCode=0 Jul 02 19:09:28.017824 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:28.017761 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" event={"ID":"e6d8b20e-5192-482a-af1f-33e3ba49ec74","Type":"ContainerDied","Data":"56cb846b537a577c29549c55c94ea74453b40ea72251f72066d74aa274e21c8d"} Jul 02 19:09:29.025053 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.025017 2571 generic.go:356] "Generic (PLEG): container finished" podID="164cd298-6237-4212-aa0b-cb6004c464f2" containerID="a246f2c1b011b1447eb14f06acd29c86673b8de5ce0d891311dd33b5b2a14ac9" exitCode=0 Jul 02 19:09:29.025493 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.025105 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" event={"ID":"164cd298-6237-4212-aa0b-cb6004c464f2","Type":"ContainerDied","Data":"a246f2c1b011b1447eb14f06acd29c86673b8de5ce0d891311dd33b5b2a14ac9"} Jul 02 19:09:29.027120 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.027094 2571 generic.go:356] "Generic (PLEG): container finished" podID="80e2a77b-0851-4bc5-8ed0-c6d49ec62683" containerID="c188401215bd14fabe5fc8b890c6d68e9008b8a9a7493f6a0ebdb31364632a3a" exitCode=0 Jul 02 19:09:29.027237 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.027154 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" event={"ID":"80e2a77b-0851-4bc5-8ed0-c6d49ec62683","Type":"ContainerDied","Data":"c188401215bd14fabe5fc8b890c6d68e9008b8a9a7493f6a0ebdb31364632a3a"} Jul 02 19:09:29.029263 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.029242 2571 generic.go:356] "Generic (PLEG): container finished" podID="a378be6c-22a8-419d-b887-2619dfb8024f" containerID="65d2b0a645c00742ffd566afa4d958dd4c4addcb2b4d90a260d51c9fc57c12a2" exitCode=0 Jul 02 19:09:29.029371 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.029316 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" event={"ID":"a378be6c-22a8-419d-b887-2619dfb8024f","Type":"ContainerDied","Data":"65d2b0a645c00742ffd566afa4d958dd4c4addcb2b4d90a260d51c9fc57c12a2"} Jul 02 19:09:29.159644 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.159617 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:29.217812 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.217776 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-bundle\") pod \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " Jul 02 19:09:29.217978 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.217831 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd8cw\" (UniqueName: \"kubernetes.io/projected/e6d8b20e-5192-482a-af1f-33e3ba49ec74-kube-api-access-dd8cw\") pod \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " Jul 02 19:09:29.217978 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.217878 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-util\") pod \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\" (UID: \"e6d8b20e-5192-482a-af1f-33e3ba49ec74\") " Jul 02 19:09:29.218353 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.218327 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-bundle" (OuterVolumeSpecName: "bundle") pod "e6d8b20e-5192-482a-af1f-33e3ba49ec74" (UID: "e6d8b20e-5192-482a-af1f-33e3ba49ec74"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:09:29.219911 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.219892 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6d8b20e-5192-482a-af1f-33e3ba49ec74-kube-api-access-dd8cw" (OuterVolumeSpecName: "kube-api-access-dd8cw") pod "e6d8b20e-5192-482a-af1f-33e3ba49ec74" (UID: "e6d8b20e-5192-482a-af1f-33e3ba49ec74"). InnerVolumeSpecName "kube-api-access-dd8cw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:09:29.223481 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.223461 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-util" (OuterVolumeSpecName: "util") pod "e6d8b20e-5192-482a-af1f-33e3ba49ec74" (UID: "e6d8b20e-5192-482a-af1f-33e3ba49ec74"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:09:29.319198 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.319117 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-bundle\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:29.319198 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.319146 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd8cw\" (UniqueName: \"kubernetes.io/projected/e6d8b20e-5192-482a-af1f-33e3ba49ec74-kube-api-access-dd8cw\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:29.319198 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:29.319159 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6d8b20e-5192-482a-af1f-33e3ba49ec74-util\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:30.035720 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.035696 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" Jul 02 19:09:30.036152 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.035725 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm" event={"ID":"e6d8b20e-5192-482a-af1f-33e3ba49ec74","Type":"ContainerDied","Data":"20519b99c82990b6148aa20f82eb3c1e6b8b1f596f9f75dde1f7cbe1fae99785"} Jul 02 19:09:30.036152 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.035779 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20519b99c82990b6148aa20f82eb3c1e6b8b1f596f9f75dde1f7cbe1fae99785" Jul 02 19:09:30.186786 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.186755 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:30.217754 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.217726 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:30.220579 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.220557 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:30.228192 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.228172 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-util\") pod \"a378be6c-22a8-419d-b887-2619dfb8024f\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " Jul 02 19:09:30.228296 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.228250 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-bundle\") pod \"a378be6c-22a8-419d-b887-2619dfb8024f\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " Jul 02 19:09:30.228340 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.228303 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49t4j\" (UniqueName: \"kubernetes.io/projected/a378be6c-22a8-419d-b887-2619dfb8024f-kube-api-access-49t4j\") pod \"a378be6c-22a8-419d-b887-2619dfb8024f\" (UID: \"a378be6c-22a8-419d-b887-2619dfb8024f\") " Jul 02 19:09:30.228843 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.228819 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-bundle" (OuterVolumeSpecName: "bundle") pod "a378be6c-22a8-419d-b887-2619dfb8024f" (UID: "a378be6c-22a8-419d-b887-2619dfb8024f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:09:30.230425 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.230404 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a378be6c-22a8-419d-b887-2619dfb8024f-kube-api-access-49t4j" (OuterVolumeSpecName: "kube-api-access-49t4j") pod "a378be6c-22a8-419d-b887-2619dfb8024f" (UID: "a378be6c-22a8-419d-b887-2619dfb8024f"). InnerVolumeSpecName "kube-api-access-49t4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:09:30.234276 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.234245 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-util" (OuterVolumeSpecName: "util") pod "a378be6c-22a8-419d-b887-2619dfb8024f" (UID: "a378be6c-22a8-419d-b887-2619dfb8024f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:09:30.329175 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.329049 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsqlx\" (UniqueName: \"kubernetes.io/projected/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-kube-api-access-qsqlx\") pod \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " Jul 02 19:09:30.329175 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.329163 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-bundle\") pod \"164cd298-6237-4212-aa0b-cb6004c464f2\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " Jul 02 19:09:30.329411 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.329197 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-util\") pod \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " Jul 02 19:09:30.329411 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.329226 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twg8q\" (UniqueName: \"kubernetes.io/projected/164cd298-6237-4212-aa0b-cb6004c464f2-kube-api-access-twg8q\") pod \"164cd298-6237-4212-aa0b-cb6004c464f2\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " Jul 02 19:09:30.329411 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.329275 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-util\") pod \"164cd298-6237-4212-aa0b-cb6004c464f2\" (UID: \"164cd298-6237-4212-aa0b-cb6004c464f2\") " Jul 02 19:09:30.329411 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.329297 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-bundle\") pod \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\" (UID: \"80e2a77b-0851-4bc5-8ed0-c6d49ec62683\") " Jul 02 19:09:30.329837 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.329753 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-bundle\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:30.329837 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.329782 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49t4j\" (UniqueName: \"kubernetes.io/projected/a378be6c-22a8-419d-b887-2619dfb8024f-kube-api-access-49t4j\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:30.329837 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.329795 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a378be6c-22a8-419d-b887-2619dfb8024f-util\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:30.330110 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.329944 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-bundle" (OuterVolumeSpecName: "bundle") pod "164cd298-6237-4212-aa0b-cb6004c464f2" (UID: "164cd298-6237-4212-aa0b-cb6004c464f2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:09:30.330110 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.329946 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-bundle" (OuterVolumeSpecName: "bundle") pod "80e2a77b-0851-4bc5-8ed0-c6d49ec62683" (UID: "80e2a77b-0851-4bc5-8ed0-c6d49ec62683"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:09:30.331304 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.331281 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-kube-api-access-qsqlx" (OuterVolumeSpecName: "kube-api-access-qsqlx") pod "80e2a77b-0851-4bc5-8ed0-c6d49ec62683" (UID: "80e2a77b-0851-4bc5-8ed0-c6d49ec62683"). InnerVolumeSpecName "kube-api-access-qsqlx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:09:30.331642 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.331623 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/164cd298-6237-4212-aa0b-cb6004c464f2-kube-api-access-twg8q" (OuterVolumeSpecName: "kube-api-access-twg8q") pod "164cd298-6237-4212-aa0b-cb6004c464f2" (UID: "164cd298-6237-4212-aa0b-cb6004c464f2"). InnerVolumeSpecName "kube-api-access-twg8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:09:30.335596 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.335548 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-util" (OuterVolumeSpecName: "util") pod "164cd298-6237-4212-aa0b-cb6004c464f2" (UID: "164cd298-6237-4212-aa0b-cb6004c464f2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:09:30.335812 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.335793 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-util" (OuterVolumeSpecName: "util") pod "80e2a77b-0851-4bc5-8ed0-c6d49ec62683" (UID: "80e2a77b-0851-4bc5-8ed0-c6d49ec62683"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:09:30.430526 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.430485 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-bundle\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:30.430526 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.430520 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-util\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:30.430526 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.430530 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twg8q\" (UniqueName: \"kubernetes.io/projected/164cd298-6237-4212-aa0b-cb6004c464f2-kube-api-access-twg8q\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:30.430744 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.430540 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/164cd298-6237-4212-aa0b-cb6004c464f2-util\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:30.430744 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.430549 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-bundle\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:30.430744 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:30.430556 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsqlx\" (UniqueName: \"kubernetes.io/projected/80e2a77b-0851-4bc5-8ed0-c6d49ec62683-kube-api-access-qsqlx\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:09:31.042632 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:31.042608 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" Jul 02 19:09:31.043052 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:31.042607 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh" event={"ID":"80e2a77b-0851-4bc5-8ed0-c6d49ec62683","Type":"ContainerDied","Data":"f98f53d3258586640ea5d1dd2185d62643231981e89c87d6bb3ee1efabe559a8"} Jul 02 19:09:31.043052 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:31.042728 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f98f53d3258586640ea5d1dd2185d62643231981e89c87d6bb3ee1efabe559a8" Jul 02 19:09:31.044496 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:31.044471 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" Jul 02 19:09:31.044627 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:31.044513 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5" event={"ID":"a378be6c-22a8-419d-b887-2619dfb8024f","Type":"ContainerDied","Data":"c706853e5bfadb253c814568a94dd24a8a4afa2047e0b8e861d3f78d714363fd"} Jul 02 19:09:31.044627 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:31.044560 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c706853e5bfadb253c814568a94dd24a8a4afa2047e0b8e861d3f78d714363fd" Jul 02 19:09:31.046452 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:31.046432 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" event={"ID":"164cd298-6237-4212-aa0b-cb6004c464f2","Type":"ContainerDied","Data":"c7fdd2b02f0778ab86495dfbca08f981db4a59546e4041147ad2539e43a2f5f9"} Jul 02 19:09:31.046572 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:31.046454 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz" Jul 02 19:09:31.046572 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:31.046467 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7fdd2b02f0778ab86495dfbca08f981db4a59546e4041147ad2539e43a2f5f9" Jul 02 19:09:40.801133 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801101 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk"] Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801426 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d8b20e-5192-482a-af1f-33e3ba49ec74" containerName="util" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801436 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d8b20e-5192-482a-af1f-33e3ba49ec74" containerName="util" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801445 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a378be6c-22a8-419d-b887-2619dfb8024f" containerName="pull" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801451 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a378be6c-22a8-419d-b887-2619dfb8024f" containerName="pull" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801459 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="164cd298-6237-4212-aa0b-cb6004c464f2" containerName="pull" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801465 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="164cd298-6237-4212-aa0b-cb6004c464f2" containerName="pull" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801471 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a378be6c-22a8-419d-b887-2619dfb8024f" containerName="extract" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801476 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a378be6c-22a8-419d-b887-2619dfb8024f" containerName="extract" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801485 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80e2a77b-0851-4bc5-8ed0-c6d49ec62683" containerName="pull" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801490 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="80e2a77b-0851-4bc5-8ed0-c6d49ec62683" containerName="pull" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801496 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="164cd298-6237-4212-aa0b-cb6004c464f2" containerName="extract" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801502 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="164cd298-6237-4212-aa0b-cb6004c464f2" containerName="extract" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801507 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80e2a77b-0851-4bc5-8ed0-c6d49ec62683" containerName="extract" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801512 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="80e2a77b-0851-4bc5-8ed0-c6d49ec62683" containerName="extract" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801520 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="164cd298-6237-4212-aa0b-cb6004c464f2" containerName="util" Jul 02 19:09:40.801518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801524 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="164cd298-6237-4212-aa0b-cb6004c464f2" containerName="util" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801529 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d8b20e-5192-482a-af1f-33e3ba49ec74" containerName="extract" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801534 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d8b20e-5192-482a-af1f-33e3ba49ec74" containerName="extract" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801541 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d8b20e-5192-482a-af1f-33e3ba49ec74" containerName="pull" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801546 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d8b20e-5192-482a-af1f-33e3ba49ec74" containerName="pull" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801552 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a378be6c-22a8-419d-b887-2619dfb8024f" containerName="util" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801556 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a378be6c-22a8-419d-b887-2619dfb8024f" containerName="util" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801562 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80e2a77b-0851-4bc5-8ed0-c6d49ec62683" containerName="util" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801567 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="80e2a77b-0851-4bc5-8ed0-c6d49ec62683" containerName="util" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801619 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a378be6c-22a8-419d-b887-2619dfb8024f" containerName="extract" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801628 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="80e2a77b-0851-4bc5-8ed0-c6d49ec62683" containerName="extract" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801635 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="164cd298-6237-4212-aa0b-cb6004c464f2" containerName="extract" Jul 02 19:09:40.801959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.801642 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6d8b20e-5192-482a-af1f-33e3ba49ec74" containerName="extract" Jul 02 19:09:40.809513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.809496 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk" Jul 02 19:09:40.813008 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.812992 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-nm8hh\"" Jul 02 19:09:40.826166 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.826141 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk"] Jul 02 19:09:40.926624 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:40.926583 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgkrc\" (UniqueName: \"kubernetes.io/projected/e874b86d-4fd3-48da-9d47-474a6bc99de4-kube-api-access-xgkrc\") pod \"authorino-operator-75cc8d5cb7-6xvfk\" (UID: \"e874b86d-4fd3-48da-9d47-474a6bc99de4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk" Jul 02 19:09:41.027530 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:41.027491 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgkrc\" (UniqueName: \"kubernetes.io/projected/e874b86d-4fd3-48da-9d47-474a6bc99de4-kube-api-access-xgkrc\") pod \"authorino-operator-75cc8d5cb7-6xvfk\" (UID: \"e874b86d-4fd3-48da-9d47-474a6bc99de4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk" Jul 02 19:09:41.038868 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:41.038836 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgkrc\" (UniqueName: \"kubernetes.io/projected/e874b86d-4fd3-48da-9d47-474a6bc99de4-kube-api-access-xgkrc\") pod \"authorino-operator-75cc8d5cb7-6xvfk\" (UID: \"e874b86d-4fd3-48da-9d47-474a6bc99de4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk" Jul 02 19:09:41.118944 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:41.118858 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk" Jul 02 19:09:41.264493 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:41.264467 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk"] Jul 02 19:09:41.267123 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:09:41.267079 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode874b86d_4fd3_48da_9d47_474a6bc99de4.slice/crio-920caf75e7cb2170e7ae6328443faede703415cfdf955d48360ecf7b7ee9af35 WatchSource:0}: Error finding container 920caf75e7cb2170e7ae6328443faede703415cfdf955d48360ecf7b7ee9af35: Status 404 returned error can't find the container with id 920caf75e7cb2170e7ae6328443faede703415cfdf955d48360ecf7b7ee9af35 Jul 02 19:09:42.109690 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:42.109636 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk" event={"ID":"e874b86d-4fd3-48da-9d47-474a6bc99de4","Type":"ContainerStarted","Data":"920caf75e7cb2170e7ae6328443faede703415cfdf955d48360ecf7b7ee9af35"} Jul 02 19:09:44.230269 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:44.230235 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf"] Jul 02 19:09:44.233759 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:44.233739 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf" Jul 02 19:09:44.237620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:44.237598 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-xzw5t\"" Jul 02 19:09:44.238044 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:44.238027 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 02 19:09:44.290704 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:44.290664 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf"] Jul 02 19:09:44.356040 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:44.355989 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck7vx\" (UniqueName: \"kubernetes.io/projected/52da5f25-6026-4f0f-829c-30d22e41ccc4-kube-api-access-ck7vx\") pod \"dns-operator-controller-manager-864dd78c4c-qb9bf\" (UID: \"52da5f25-6026-4f0f-829c-30d22e41ccc4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf" Jul 02 19:09:44.456603 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:44.456568 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck7vx\" (UniqueName: \"kubernetes.io/projected/52da5f25-6026-4f0f-829c-30d22e41ccc4-kube-api-access-ck7vx\") pod \"dns-operator-controller-manager-864dd78c4c-qb9bf\" (UID: \"52da5f25-6026-4f0f-829c-30d22e41ccc4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf" Jul 02 19:09:44.474774 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:44.474740 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck7vx\" (UniqueName: \"kubernetes.io/projected/52da5f25-6026-4f0f-829c-30d22e41ccc4-kube-api-access-ck7vx\") pod \"dns-operator-controller-manager-864dd78c4c-qb9bf\" (UID: \"52da5f25-6026-4f0f-829c-30d22e41ccc4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf" Jul 02 19:09:44.543708 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:44.543674 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf" Jul 02 19:09:44.687039 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:44.687010 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf"] Jul 02 19:09:44.688874 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:09:44.688848 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52da5f25_6026_4f0f_829c_30d22e41ccc4.slice/crio-f8d361d82b8cba09023c8c1c143ce08a2725505275a5f9ffb3a43a44925af47d WatchSource:0}: Error finding container f8d361d82b8cba09023c8c1c143ce08a2725505275a5f9ffb3a43a44925af47d: Status 404 returned error can't find the container with id f8d361d82b8cba09023c8c1c143ce08a2725505275a5f9ffb3a43a44925af47d Jul 02 19:09:45.137369 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:45.137334 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk" event={"ID":"e874b86d-4fd3-48da-9d47-474a6bc99de4","Type":"ContainerStarted","Data":"205bd1a8c754d3a79c823861dfdeac1774d789d997153c817de27de03d157621"} Jul 02 19:09:45.137548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:45.137433 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk" Jul 02 19:09:45.138657 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:45.138635 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf" event={"ID":"52da5f25-6026-4f0f-829c-30d22e41ccc4","Type":"ContainerStarted","Data":"f8d361d82b8cba09023c8c1c143ce08a2725505275a5f9ffb3a43a44925af47d"} Jul 02 19:09:45.159191 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:45.159141 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk" podStartSLOduration=1.82792582 podStartE2EDuration="5.159125851s" totalImagesPullingTime="3.331200031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:45.157605678 +0000 UTC m=+422.448942581" watchObservedRunningTime="2026-07-02 19:09:45.159125851 +0000 UTC m=+422.450462751" Jul 02 19:09:47.961510 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:47.961470 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l"] Jul 02 19:09:47.965152 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:47.965133 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:09:47.968105 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:47.968065 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-9kgpw\"" Jul 02 19:09:47.983519 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:47.983492 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l"] Jul 02 19:09:48.089634 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:48.089600 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5k24\" (UniqueName: \"kubernetes.io/projected/34e96e33-36cf-432c-90e3-c6401989187f-kube-api-access-h5k24\") pod \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" (UID: \"34e96e33-36cf-432c-90e3-c6401989187f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:09:48.162402 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:48.162362 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf" event={"ID":"52da5f25-6026-4f0f-829c-30d22e41ccc4","Type":"ContainerStarted","Data":"2c5995f6138f96ce067aba6cf5e651e05355ce66271873c8759d3eed76e06fed"} Jul 02 19:09:48.162605 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:48.162533 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf" Jul 02 19:09:48.190776 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:48.190736 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5k24\" (UniqueName: \"kubernetes.io/projected/34e96e33-36cf-432c-90e3-c6401989187f-kube-api-access-h5k24\") pod \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" (UID: \"34e96e33-36cf-432c-90e3-c6401989187f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:09:48.193818 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:48.193769 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf" podStartSLOduration=1.8129060830000001 podStartE2EDuration="4.193755153s" totalImagesPullingTime="2.38084907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:48.192088441 +0000 UTC m=+425.483425341" watchObservedRunningTime="2026-07-02 19:09:48.193755153 +0000 UTC m=+425.485092054" Jul 02 19:09:48.217651 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:48.217592 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5k24\" (UniqueName: \"kubernetes.io/projected/34e96e33-36cf-432c-90e3-c6401989187f-kube-api-access-h5k24\") pod \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" (UID: \"34e96e33-36cf-432c-90e3-c6401989187f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:09:48.275211 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:48.275180 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:09:48.422972 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:48.422944 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l"] Jul 02 19:09:48.425155 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:09:48.425127 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34e96e33_36cf_432c_90e3_c6401989187f.slice/crio-c02732ad2762d32daf4d1c456cae5479c92f070889f8e59d1d0ca850b4acc2b7 WatchSource:0}: Error finding container c02732ad2762d32daf4d1c456cae5479c92f070889f8e59d1d0ca850b4acc2b7: Status 404 returned error can't find the container with id c02732ad2762d32daf4d1c456cae5479c92f070889f8e59d1d0ca850b4acc2b7 Jul 02 19:09:49.171357 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:49.171318 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" event={"ID":"34e96e33-36cf-432c-90e3-c6401989187f","Type":"ContainerStarted","Data":"c02732ad2762d32daf4d1c456cae5479c92f070889f8e59d1d0ca850b4acc2b7"} Jul 02 19:09:51.183983 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:51.183947 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" event={"ID":"34e96e33-36cf-432c-90e3-c6401989187f","Type":"ContainerStarted","Data":"6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f"} Jul 02 19:09:51.184418 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:51.184009 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:09:51.206152 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:51.206096 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" podStartSLOduration=2.084170729 podStartE2EDuration="4.206081046s" totalImagesPullingTime="2.121910317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:51.203847243 +0000 UTC m=+428.495184144" watchObservedRunningTime="2026-07-02 19:09:51.206081046 +0000 UTC m=+428.497417936" Jul 02 19:09:53.396453 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.396422 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm"] Jul 02 19:09:53.399915 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.399896 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:09:53.402660 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.402631 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z4p6g\"" Jul 02 19:09:53.419065 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.419039 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm"] Jul 02 19:09:53.548377 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.548343 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9da0b67-cb6d-448a-8614-032e087ee72d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" (UID: \"b9da0b67-cb6d-448a-8614-032e087ee72d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:09:53.548377 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.548381 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5sm8\" (UniqueName: \"kubernetes.io/projected/b9da0b67-cb6d-448a-8614-032e087ee72d-kube-api-access-h5sm8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" (UID: \"b9da0b67-cb6d-448a-8614-032e087ee72d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:09:53.649935 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.649831 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9da0b67-cb6d-448a-8614-032e087ee72d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" (UID: \"b9da0b67-cb6d-448a-8614-032e087ee72d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:09:53.649935 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.649884 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5sm8\" (UniqueName: \"kubernetes.io/projected/b9da0b67-cb6d-448a-8614-032e087ee72d-kube-api-access-h5sm8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" (UID: \"b9da0b67-cb6d-448a-8614-032e087ee72d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:09:53.650279 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.650258 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9da0b67-cb6d-448a-8614-032e087ee72d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" (UID: \"b9da0b67-cb6d-448a-8614-032e087ee72d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:09:53.665394 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.665362 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5sm8\" (UniqueName: \"kubernetes.io/projected/b9da0b67-cb6d-448a-8614-032e087ee72d-kube-api-access-h5sm8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" (UID: \"b9da0b67-cb6d-448a-8614-032e087ee72d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:09:53.710463 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.710428 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:09:53.852129 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:53.852100 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm"] Jul 02 19:09:53.853877 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:09:53.853856 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9da0b67_cb6d_448a_8614_032e087ee72d.slice/crio-dd8212b06bf5308a6e059ada07baef9f5c8501f668a1f85e8cbaddd9fd18281e WatchSource:0}: Error finding container dd8212b06bf5308a6e059ada07baef9f5c8501f668a1f85e8cbaddd9fd18281e: Status 404 returned error can't find the container with id dd8212b06bf5308a6e059ada07baef9f5c8501f668a1f85e8cbaddd9fd18281e Jul 02 19:09:54.200591 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:54.200558 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" event={"ID":"b9da0b67-cb6d-448a-8614-032e087ee72d","Type":"ContainerStarted","Data":"dd8212b06bf5308a6e059ada07baef9f5c8501f668a1f85e8cbaddd9fd18281e"} Jul 02 19:09:56.151321 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:56.151294 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6xvfk" Jul 02 19:09:59.174090 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:09:59.174049 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9bf" Jul 02 19:10:01.246561 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:01.246505 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" event={"ID":"b9da0b67-cb6d-448a-8614-032e087ee72d","Type":"ContainerStarted","Data":"eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62"} Jul 02 19:10:01.246977 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:01.246638 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:01.267915 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:01.267861 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" podStartSLOduration=1.541389849 podStartE2EDuration="8.26784304s" totalImagesPullingTime="6.726453191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:10:01.265845765 +0000 UTC m=+438.557182665" watchObservedRunningTime="2026-07-02 19:10:01.26784304 +0000 UTC m=+438.559179939" Jul 02 19:10:02.191779 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:02.191747 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:03.452859 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.452817 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8"] Jul 02 19:10:03.458053 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.458026 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" Jul 02 19:10:03.460465 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.460440 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 02 19:10:03.460607 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.460440 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-s9r5w\"" Jul 02 19:10:03.460681 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.460611 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 02 19:10:03.467458 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.467418 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8"] Jul 02 19:10:03.536041 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.536008 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0acce477-a26f-4073-b651-8bcfc6b7cbc9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b5jp8\" (UID: \"0acce477-a26f-4073-b651-8bcfc6b7cbc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" Jul 02 19:10:03.536253 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.536169 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0acce477-a26f-4073-b651-8bcfc6b7cbc9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b5jp8\" (UID: \"0acce477-a26f-4073-b651-8bcfc6b7cbc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" Jul 02 19:10:03.536253 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.536195 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8hzx\" (UniqueName: \"kubernetes.io/projected/0acce477-a26f-4073-b651-8bcfc6b7cbc9-kube-api-access-f8hzx\") pod \"kuadrant-console-plugin-66b5774985-b5jp8\" (UID: \"0acce477-a26f-4073-b651-8bcfc6b7cbc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" Jul 02 19:10:03.636847 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.636807 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0acce477-a26f-4073-b651-8bcfc6b7cbc9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b5jp8\" (UID: \"0acce477-a26f-4073-b651-8bcfc6b7cbc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" Jul 02 19:10:03.636847 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.636845 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8hzx\" (UniqueName: \"kubernetes.io/projected/0acce477-a26f-4073-b651-8bcfc6b7cbc9-kube-api-access-f8hzx\") pod \"kuadrant-console-plugin-66b5774985-b5jp8\" (UID: \"0acce477-a26f-4073-b651-8bcfc6b7cbc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" Jul 02 19:10:03.637117 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.636884 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0acce477-a26f-4073-b651-8bcfc6b7cbc9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b5jp8\" (UID: \"0acce477-a26f-4073-b651-8bcfc6b7cbc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" Jul 02 19:10:03.637640 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.637611 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0acce477-a26f-4073-b651-8bcfc6b7cbc9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b5jp8\" (UID: \"0acce477-a26f-4073-b651-8bcfc6b7cbc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" Jul 02 19:10:03.639423 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.639393 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0acce477-a26f-4073-b651-8bcfc6b7cbc9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b5jp8\" (UID: \"0acce477-a26f-4073-b651-8bcfc6b7cbc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" Jul 02 19:10:03.645534 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.645502 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8hzx\" (UniqueName: \"kubernetes.io/projected/0acce477-a26f-4073-b651-8bcfc6b7cbc9-kube-api-access-f8hzx\") pod \"kuadrant-console-plugin-66b5774985-b5jp8\" (UID: \"0acce477-a26f-4073-b651-8bcfc6b7cbc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" Jul 02 19:10:03.769050 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.769019 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" Jul 02 19:10:03.914228 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:03.914193 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8"] Jul 02 19:10:03.916249 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:10:03.916223 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0acce477_a26f_4073_b651_8bcfc6b7cbc9.slice/crio-d8aa7c08868f6dd0d810e9e983956d7108119bddcccff9fbd43a96941267e97a WatchSource:0}: Error finding container d8aa7c08868f6dd0d810e9e983956d7108119bddcccff9fbd43a96941267e97a: Status 404 returned error can't find the container with id d8aa7c08868f6dd0d810e9e983956d7108119bddcccff9fbd43a96941267e97a Jul 02 19:10:04.264879 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:04.264846 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" event={"ID":"0acce477-a26f-4073-b651-8bcfc6b7cbc9","Type":"ContainerStarted","Data":"d8aa7c08868f6dd0d810e9e983956d7108119bddcccff9fbd43a96941267e97a"} Jul 02 19:10:12.255703 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:12.255670 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:14.538975 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.538939 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm"] Jul 02 19:10:14.539422 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.539173 2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" containerName="manager" containerID="cri-o://eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62" gracePeriod=2 Jul 02 19:10:14.545135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.545105 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm"] Jul 02 19:10:14.546271 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:14.546233 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:14.568467 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.568433 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l"] Jul 02 19:10:14.568697 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.568668 2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" podUID="34e96e33-36cf-432c-90e3-c6401989187f" containerName="manager" containerID="cri-o://6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f" gracePeriod=2 Jul 02 19:10:14.575356 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:14.575325 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:14.577940 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:14.577899 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="34e96e33-36cf-432c-90e3-c6401989187f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:14.578407 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.578317 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l"] Jul 02 19:10:14.620221 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.620187 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r"] Jul 02 19:10:14.620984 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.620559 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" containerName="manager" Jul 02 19:10:14.620984 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.620609 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" containerName="manager" Jul 02 19:10:14.620984 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.620623 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34e96e33-36cf-432c-90e3-c6401989187f" containerName="manager" Jul 02 19:10:14.620984 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.620633 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="34e96e33-36cf-432c-90e3-c6401989187f" containerName="manager" Jul 02 19:10:14.620984 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.620732 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" containerName="manager" Jul 02 19:10:14.620984 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.620744 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="34e96e33-36cf-432c-90e3-c6401989187f" containerName="manager" Jul 02 19:10:14.627723 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.627695 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj"] Jul 02 19:10:14.627876 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.627833 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:14.630885 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:14.630851 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="34e96e33-36cf-432c-90e3-c6401989187f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:14.632806 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.632787 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj" Jul 02 19:10:14.656829 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.656791 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj"] Jul 02 19:10:14.672519 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.672486 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r"] Jul 02 19:10:14.735414 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.735355 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l8m7r\" (UID: \"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:14.735666 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.735449 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mmk9\" (UniqueName: \"kubernetes.io/projected/3f0232b3-b880-4da1-9b04-6d1c62e637de-kube-api-access-7mmk9\") pod \"limitador-operator-controller-manager-859d7b78bd-9r9fj\" (UID: \"3f0232b3-b880-4da1-9b04-6d1c62e637de\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj" Jul 02 19:10:14.735666 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.735482 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w64q\" (UniqueName: \"kubernetes.io/projected/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-kube-api-access-7w64q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l8m7r\" (UID: \"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:14.738686 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:14.738648 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:14.740856 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:14.740821 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:14.780347 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:14.780306 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="34e96e33-36cf-432c-90e3-c6401989187f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:14.836959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.836872 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l8m7r\" (UID: \"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:14.836959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.836935 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mmk9\" (UniqueName: \"kubernetes.io/projected/3f0232b3-b880-4da1-9b04-6d1c62e637de-kube-api-access-7mmk9\") pod \"limitador-operator-controller-manager-859d7b78bd-9r9fj\" (UID: \"3f0232b3-b880-4da1-9b04-6d1c62e637de\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj" Jul 02 19:10:14.837210 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.836967 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w64q\" (UniqueName: \"kubernetes.io/projected/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-kube-api-access-7w64q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l8m7r\" (UID: \"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:14.837368 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.837328 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l8m7r\" (UID: \"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:14.873103 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.873048 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mmk9\" (UniqueName: \"kubernetes.io/projected/3f0232b3-b880-4da1-9b04-6d1c62e637de-kube-api-access-7mmk9\") pod \"limitador-operator-controller-manager-859d7b78bd-9r9fj\" (UID: \"3f0232b3-b880-4da1-9b04-6d1c62e637de\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj" Jul 02 19:10:14.875115 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.875065 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w64q\" (UniqueName: \"kubernetes.io/projected/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-kube-api-access-7w64q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l8m7r\" (UID: \"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:14.954263 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.954222 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:14.963200 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:14.963171 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj" Jul 02 19:10:27.382489 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.382462 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:27.385683 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.385597 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:27.385978 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.385937 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="34e96e33-36cf-432c-90e3-c6401989187f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:27.388555 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.388396 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:27.389581 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.389559 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r"] Jul 02 19:10:27.390647 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:10:27.390621 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a12645e_713e_4a94_8a0d_6cc7f3cd3a02.slice/crio-f8d245dfecf6fba318a4b982c95f144afb97891199c5589b069f50b57b70bc6d WatchSource:0}: Error finding container f8d245dfecf6fba318a4b982c95f144afb97891199c5589b069f50b57b70bc6d: Status 404 returned error can't find the container with id f8d245dfecf6fba318a4b982c95f144afb97891199c5589b069f50b57b70bc6d Jul 02 19:10:27.393033 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.393006 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="34e96e33-36cf-432c-90e3-c6401989187f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:27.410624 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.410589 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" event={"ID":"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02","Type":"ContainerStarted","Data":"f8d245dfecf6fba318a4b982c95f144afb97891199c5589b069f50b57b70bc6d"} Jul 02 19:10:27.412326 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.412294 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" event={"ID":"0acce477-a26f-4073-b651-8bcfc6b7cbc9","Type":"ContainerStarted","Data":"a4604aba7647f41e0b5263a4e434f1784ca2121552ca747330a611365bf5be88"} Jul 02 19:10:27.413689 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.413667 2571 generic.go:356] "Generic (PLEG): container finished" podID="b9da0b67-cb6d-448a-8614-032e087ee72d" containerID="eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62" exitCode=0 Jul 02 19:10:27.413784 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.413711 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:27.413784 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.413760 2571 scope.go:117] "RemoveContainer" containerID="eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62" Jul 02 19:10:27.414734 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.414659 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:27.414991 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.414973 2571 generic.go:356] "Generic (PLEG): container finished" podID="34e96e33-36cf-432c-90e3-c6401989187f" containerID="6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f" exitCode=0 Jul 02 19:10:27.415157 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.415029 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:27.422329 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.422300 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj"] Jul 02 19:10:27.425296 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.425272 2571 scope.go:117] "RemoveContainer" containerID="eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62" Jul 02 19:10:27.425603 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.425586 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62\": container with ID starting with eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62 not found: ID does not exist" containerID="eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62" Jul 02 19:10:27.425658 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.425611 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62"} err="failed to get container status \"eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62\": rpc error: code = NotFound desc = could not find container \"eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62\": container with ID starting with eefd1b59698d970fb358301638fde7fff1a70cb1cdc4951d9527bd7aa1cc3a62 not found: ID does not exist" Jul 02 19:10:27.425658 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.425627 2571 scope.go:117] "RemoveContainer" containerID="6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f" Jul 02 19:10:27.437459 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:10:27.437407 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0232b3_b880_4da1_9b04_6d1c62e637de.slice/crio-c2a09487df7d3fb7eda92a9ac78ed71bbdd7c5e4bca89e7e25ce649b5f8a98dc WatchSource:0}: Error finding container c2a09487df7d3fb7eda92a9ac78ed71bbdd7c5e4bca89e7e25ce649b5f8a98dc: Status 404 returned error can't find the container with id c2a09487df7d3fb7eda92a9ac78ed71bbdd7c5e4bca89e7e25ce649b5f8a98dc Jul 02 19:10:27.442201 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.442158 2571 scope.go:117] "RemoveContainer" containerID="6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f" Jul 02 19:10:27.442494 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.442464 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f\": container with ID starting with 6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f not found: ID does not exist" containerID="6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f" Jul 02 19:10:27.442600 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.442506 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f"} err="failed to get container status \"6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f\": rpc error: code = NotFound desc = could not find container \"6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f\": container with ID starting with 6b14fc2bea15eccaacb3e7b4ed8954c40d884bc02d9862fc46038df3dfe65f7f not found: ID does not exist" Jul 02 19:10:27.457496 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.457460 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="34e96e33-36cf-432c-90e3-c6401989187f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:27.457647 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.457559 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5sm8\" (UniqueName: \"kubernetes.io/projected/b9da0b67-cb6d-448a-8614-032e087ee72d-kube-api-access-h5sm8\") pod \"b9da0b67-cb6d-448a-8614-032e087ee72d\" (UID: \"b9da0b67-cb6d-448a-8614-032e087ee72d\") " Jul 02 19:10:27.457647 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.457634 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5k24\" (UniqueName: \"kubernetes.io/projected/34e96e33-36cf-432c-90e3-c6401989187f-kube-api-access-h5k24\") pod \"34e96e33-36cf-432c-90e3-c6401989187f\" (UID: \"34e96e33-36cf-432c-90e3-c6401989187f\") " Jul 02 19:10:27.457770 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.457692 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9da0b67-cb6d-448a-8614-032e087ee72d-extensions-socket-volume\") pod \"b9da0b67-cb6d-448a-8614-032e087ee72d\" (UID: \"b9da0b67-cb6d-448a-8614-032e087ee72d\") " Jul 02 19:10:27.458480 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.458455 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9da0b67-cb6d-448a-8614-032e087ee72d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b9da0b67-cb6d-448a-8614-032e087ee72d" (UID: "b9da0b67-cb6d-448a-8614-032e087ee72d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:10:27.459254 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.459221 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:27.460292 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.460263 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9da0b67-cb6d-448a-8614-032e087ee72d-kube-api-access-h5sm8" (OuterVolumeSpecName: "kube-api-access-h5sm8") pod "b9da0b67-cb6d-448a-8614-032e087ee72d" (UID: "b9da0b67-cb6d-448a-8614-032e087ee72d"). InnerVolumeSpecName "kube-api-access-h5sm8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:10:27.460395 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.460370 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e96e33-36cf-432c-90e3-c6401989187f-kube-api-access-h5k24" (OuterVolumeSpecName: "kube-api-access-h5k24") pod "34e96e33-36cf-432c-90e3-c6401989187f" (UID: "34e96e33-36cf-432c-90e3-c6401989187f"). InnerVolumeSpecName "kube-api-access-h5k24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:10:27.462358 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.462306 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="34e96e33-36cf-432c-90e3-c6401989187f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:27.558883 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.558848 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5sm8\" (UniqueName: \"kubernetes.io/projected/b9da0b67-cb6d-448a-8614-032e087ee72d-kube-api-access-h5sm8\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:10:27.558883 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.558883 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5k24\" (UniqueName: \"kubernetes.io/projected/34e96e33-36cf-432c-90e3-c6401989187f-kube-api-access-h5k24\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:10:27.559117 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:27.558897 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9da0b67-cb6d-448a-8614-032e087ee72d-extensions-socket-volume\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:10:27.727088 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.727048 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="34e96e33-36cf-432c-90e3-c6401989187f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:27.731665 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.731630 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:27.734441 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.734411 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:27.737420 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:27.737397 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="34e96e33-36cf-432c-90e3-c6401989187f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:28.422205 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:28.422167 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" event={"ID":"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02","Type":"ContainerStarted","Data":"d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42"} Jul 02 19:10:28.422654 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:28.422229 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:28.425394 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:28.425371 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj" event={"ID":"3f0232b3-b880-4da1-9b04-6d1c62e637de","Type":"ContainerStarted","Data":"88691ec1657aa076d9d71d1a6632e3a219384103c11bc5665667f1a9b3bcd1cc"} Jul 02 19:10:28.425511 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:28.425408 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj" event={"ID":"3f0232b3-b880-4da1-9b04-6d1c62e637de","Type":"ContainerStarted","Data":"c2a09487df7d3fb7eda92a9ac78ed71bbdd7c5e4bca89e7e25ce649b5f8a98dc"} Jul 02 19:10:28.464626 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:28.464514 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" podStartSLOduration=14.464490027 podStartE2EDuration="14.464490027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:10:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:10:28.459530762 +0000 UTC m=+465.750867662" watchObservedRunningTime="2026-07-02 19:10:28.464490027 +0000 UTC m=+465.755826957" Jul 02 19:10:28.464626 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:28.464569 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:28.464626 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:28.464612 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5jp8" podStartSLOduration=2.106319562 podStartE2EDuration="25.464606604s" totalImagesPullingTime="23.358287042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:10:27.455489296 +0000 UTC m=+464.746826196" watchObservedRunningTime="2026-07-02 19:10:28.464606604 +0000 UTC m=+465.755943504" Jul 02 19:10:28.470202 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:28.470175 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="34e96e33-36cf-432c-90e3-c6401989187f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:28.476984 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:28.476952 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g8l6l\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="34e96e33-36cf-432c-90e3-c6401989187f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g8l6l" Jul 02 19:10:28.479089 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:28.479054 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4crwm\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4crwm" Jul 02 19:10:29.237610 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:29.237578 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34e96e33-36cf-432c-90e3-c6401989187f" path="/var/lib/kubelet/pods/34e96e33-36cf-432c-90e3-c6401989187f/volumes" Jul 02 19:10:29.237939 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:29.237926 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9da0b67-cb6d-448a-8614-032e087ee72d" path="/var/lib/kubelet/pods/b9da0b67-cb6d-448a-8614-032e087ee72d/volumes" Jul 02 19:10:29.430758 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:29.430726 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj" Jul 02 19:10:33.350196 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:33.350129 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj" podStartSLOduration=19.350105028 podStartE2EDuration="19.350105028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:10:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:10:28.523286946 +0000 UTC m=+465.814623847" watchObservedRunningTime="2026-07-02 19:10:33.350105028 +0000 UTC m=+470.641441926" Jul 02 19:10:33.352518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:33.352494 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r"] Jul 02 19:10:33.352685 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:33.352660 2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" podUID="0a12645e-713e-4a94-8a0d-6cc7f3cd3a02" containerName="manager" containerID="cri-o://d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42" gracePeriod=10 Jul 02 19:10:33.354770 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:33.354748 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:33.608421 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:33.608351 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:33.714266 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:33.714237 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-extensions-socket-volume\") pod \"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02\" (UID: \"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02\") " Jul 02 19:10:33.714437 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:33.714359 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w64q\" (UniqueName: \"kubernetes.io/projected/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-kube-api-access-7w64q\") pod \"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02\" (UID: \"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02\") " Jul 02 19:10:33.714627 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:33.714599 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0a12645e-713e-4a94-8a0d-6cc7f3cd3a02" (UID: "0a12645e-713e-4a94-8a0d-6cc7f3cd3a02"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:10:33.716377 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:33.716353 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-kube-api-access-7w64q" (OuterVolumeSpecName: "kube-api-access-7w64q") pod "0a12645e-713e-4a94-8a0d-6cc7f3cd3a02" (UID: "0a12645e-713e-4a94-8a0d-6cc7f3cd3a02"). InnerVolumeSpecName "kube-api-access-7w64q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:10:33.814947 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:33.814896 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-extensions-socket-volume\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:10:33.814947 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:33.814940 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w64q\" (UniqueName: \"kubernetes.io/projected/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02-kube-api-access-7w64q\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:10:34.457096 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:34.457043 2571 generic.go:356] "Generic (PLEG): container finished" podID="0a12645e-713e-4a94-8a0d-6cc7f3cd3a02" containerID="d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42" exitCode=0 Jul 02 19:10:34.457479 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:34.457109 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" Jul 02 19:10:34.457479 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:34.457109 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" event={"ID":"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02","Type":"ContainerDied","Data":"d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42"} Jul 02 19:10:34.457479 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:34.457175 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r" event={"ID":"0a12645e-713e-4a94-8a0d-6cc7f3cd3a02","Type":"ContainerDied","Data":"f8d245dfecf6fba318a4b982c95f144afb97891199c5589b069f50b57b70bc6d"} Jul 02 19:10:34.457479 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:34.457207 2571 scope.go:117] "RemoveContainer" containerID="d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42" Jul 02 19:10:34.465946 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:34.465927 2571 scope.go:117] "RemoveContainer" containerID="d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42" Jul 02 19:10:34.466258 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:10:34.466237 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42\": container with ID starting with d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42 not found: ID does not exist" containerID="d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42" Jul 02 19:10:34.466312 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:34.466269 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42"} err="failed to get container status \"d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42\": rpc error: code = NotFound desc = could not find container \"d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42\": container with ID starting with d67ffcf8a2a4d5e8cd2ccd066f1eb34c8246079cd40f20d0e1e141a97fde0d42 not found: ID does not exist" Jul 02 19:10:34.484200 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:34.484038 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r"] Jul 02 19:10:34.486361 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:34.486337 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l8m7r"] Jul 02 19:10:35.237428 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:35.237399 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a12645e-713e-4a94-8a0d-6cc7f3cd3a02" path="/var/lib/kubelet/pods/0a12645e-713e-4a94-8a0d-6cc7f3cd3a02/volumes" Jul 02 19:10:40.438243 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:40.438212 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9r9fj" Jul 02 19:10:49.706541 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.706500 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69"] Jul 02 19:10:49.707058 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.707033 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a12645e-713e-4a94-8a0d-6cc7f3cd3a02" containerName="manager" Jul 02 19:10:49.707058 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.707059 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a12645e-713e-4a94-8a0d-6cc7f3cd3a02" containerName="manager" Jul 02 19:10:49.707282 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.707180 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a12645e-713e-4a94-8a0d-6cc7f3cd3a02" containerName="manager" Jul 02 19:10:49.820722 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.820684 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69"] Jul 02 19:10:49.820894 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.820785 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.823133 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.823109 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 19:10:49.823266 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.823222 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-txrlk\"" Jul 02 19:10:49.847765 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.847730 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.847765 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.847766 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2515057-7616-44fa-8b3c-5878008819af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.847977 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.847788 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwbzp\" (UniqueName: \"kubernetes.io/projected/d2515057-7616-44fa-8b3c-5878008819af-kube-api-access-jwbzp\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.847977 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.847815 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.847977 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.847871 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.847977 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.847895 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.847977 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.847912 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2515057-7616-44fa-8b3c-5878008819af-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.847977 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.847934 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2515057-7616-44fa-8b3c-5878008819af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.847977 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.847963 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.948445 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.948410 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.948445 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.948448 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.948681 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.948463 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2515057-7616-44fa-8b3c-5878008819af-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.948681 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.948487 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2515057-7616-44fa-8b3c-5878008819af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.948681 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.948530 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.948681 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.948573 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.948681 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.948596 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2515057-7616-44fa-8b3c-5878008819af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.948681 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.948623 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwbzp\" (UniqueName: \"kubernetes.io/projected/d2515057-7616-44fa-8b3c-5878008819af-kube-api-access-jwbzp\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.948681 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.948661 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.949034 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.948836 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.949034 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.949000 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.949034 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.949017 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.949226 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.949057 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.949334 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.949313 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2515057-7616-44fa-8b3c-5878008819af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.950727 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.950704 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2515057-7616-44fa-8b3c-5878008819af-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.951246 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.951229 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2515057-7616-44fa-8b3c-5878008819af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.955764 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.955741 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2515057-7616-44fa-8b3c-5878008819af-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:49.956043 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:49.956026 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwbzp\" (UniqueName: \"kubernetes.io/projected/d2515057-7616-44fa-8b3c-5878008819af-kube-api-access-jwbzp\") pod \"maas-default-gateway-openshift-default-56654dcf7-bqr69\" (UID: \"d2515057-7616-44fa-8b3c-5878008819af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:50.132020 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:50.131978 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:50.318663 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:50.318639 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69"] Jul 02 19:10:50.320482 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:10:50.320454 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2515057_7616_44fa_8b3c_5878008819af.slice/crio-acbf03f45fa49d6b96c8db1c279e572db724771d352fc42fd00953c62c4c411b WatchSource:0}: Error finding container acbf03f45fa49d6b96c8db1c279e572db724771d352fc42fd00953c62c4c411b: Status 404 returned error can't find the container with id acbf03f45fa49d6b96c8db1c279e572db724771d352fc42fd00953c62c4c411b Jul 02 19:10:50.544355 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:50.544321 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" event={"ID":"d2515057-7616-44fa-8b3c-5878008819af","Type":"ContainerStarted","Data":"acbf03f45fa49d6b96c8db1c279e572db724771d352fc42fd00953c62c4c411b"} Jul 02 19:10:53.497859 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:53.497811 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:10:53.498295 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:53.497902 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:10:53.498295 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:53.497949 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:10:54.571965 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:54.571932 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" event={"ID":"d2515057-7616-44fa-8b3c-5878008819af","Type":"ContainerStarted","Data":"bc930632f6b33dc9be969877df39841466fdb50e12c78c6e38bb25b8cedbe361"} Jul 02 19:10:54.604348 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:54.601811 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" podStartSLOduration=2.427048539 podStartE2EDuration="5.601791478s" totalImagesPullingTime="3.174742939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:10:54.600606699 +0000 UTC m=+491.891943599" watchObservedRunningTime="2026-07-02 19:10:54.601791478 +0000 UTC m=+491.893128377" Jul 02 19:10:55.132998 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:55.132948 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:55.138018 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:55.137993 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:55.577674 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:55.577646 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:55.578698 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:55.578675 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:10:55.717562 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:55.717531 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sz8w8"] Jul 02 19:10:55.878630 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:55.878552 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sz8w8"] Jul 02 19:10:55.878630 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:55.878592 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sz8w8"] Jul 02 19:10:55.878849 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:55.878692 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:10:55.881280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:55.881256 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 19:10:55.900721 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:55.900690 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/80eddcda-1461-401a-9bb2-ba5e3f05f52d-config-file\") pod \"limitador-limitador-64c94fd64f-sz8w8\" (UID: \"80eddcda-1461-401a-9bb2-ba5e3f05f52d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:10:55.900880 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:55.900795 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr5pb\" (UniqueName: \"kubernetes.io/projected/80eddcda-1461-401a-9bb2-ba5e3f05f52d-kube-api-access-rr5pb\") pod \"limitador-limitador-64c94fd64f-sz8w8\" (UID: \"80eddcda-1461-401a-9bb2-ba5e3f05f52d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:10:56.001444 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.001402 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/80eddcda-1461-401a-9bb2-ba5e3f05f52d-config-file\") pod \"limitador-limitador-64c94fd64f-sz8w8\" (UID: \"80eddcda-1461-401a-9bb2-ba5e3f05f52d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:10:56.001602 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.001501 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr5pb\" (UniqueName: \"kubernetes.io/projected/80eddcda-1461-401a-9bb2-ba5e3f05f52d-kube-api-access-rr5pb\") pod \"limitador-limitador-64c94fd64f-sz8w8\" (UID: \"80eddcda-1461-401a-9bb2-ba5e3f05f52d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:10:56.002042 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.002018 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/80eddcda-1461-401a-9bb2-ba5e3f05f52d-config-file\") pod \"limitador-limitador-64c94fd64f-sz8w8\" (UID: \"80eddcda-1461-401a-9bb2-ba5e3f05f52d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:10:56.010865 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.010837 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr5pb\" (UniqueName: \"kubernetes.io/projected/80eddcda-1461-401a-9bb2-ba5e3f05f52d-kube-api-access-rr5pb\") pod \"limitador-limitador-64c94fd64f-sz8w8\" (UID: \"80eddcda-1461-401a-9bb2-ba5e3f05f52d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:10:56.189200 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.189124 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:10:56.396201 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.396143 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sz8w8"] Jul 02 19:10:56.398526 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:10:56.398499 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80eddcda_1461_401a_9bb2_ba5e3f05f52d.slice/crio-a82e279b8e39798d0b24be54432d116eb8be2ab1babe7e9c7dd5284e3055c28d WatchSource:0}: Error finding container a82e279b8e39798d0b24be54432d116eb8be2ab1babe7e9c7dd5284e3055c28d: Status 404 returned error can't find the container with id a82e279b8e39798d0b24be54432d116eb8be2ab1babe7e9c7dd5284e3055c28d Jul 02 19:10:56.535098 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.535041 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-s5t54"] Jul 02 19:10:56.569604 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.569561 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s5t54"] Jul 02 19:10:56.569790 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.569694 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s5t54" Jul 02 19:10:56.572490 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.572464 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7pj22\"" Jul 02 19:10:56.583541 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.583514 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" event={"ID":"80eddcda-1461-401a-9bb2-ba5e3f05f52d","Type":"ContainerStarted","Data":"a82e279b8e39798d0b24be54432d116eb8be2ab1babe7e9c7dd5284e3055c28d"} Jul 02 19:10:56.607143 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.607112 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5bk7\" (UniqueName: \"kubernetes.io/projected/cb7e1ec8-4413-4f12-aa67-597524a328b4-kube-api-access-l5bk7\") pod \"authorino-58cd955675-s5t54\" (UID: \"cb7e1ec8-4413-4f12-aa67-597524a328b4\") " pod="kuadrant-system/authorino-58cd955675-s5t54" Jul 02 19:10:56.708707 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.708671 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5bk7\" (UniqueName: \"kubernetes.io/projected/cb7e1ec8-4413-4f12-aa67-597524a328b4-kube-api-access-l5bk7\") pod \"authorino-58cd955675-s5t54\" (UID: \"cb7e1ec8-4413-4f12-aa67-597524a328b4\") " pod="kuadrant-system/authorino-58cd955675-s5t54" Jul 02 19:10:56.720715 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.720688 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5bk7\" (UniqueName: \"kubernetes.io/projected/cb7e1ec8-4413-4f12-aa67-597524a328b4-kube-api-access-l5bk7\") pod \"authorino-58cd955675-s5t54\" (UID: \"cb7e1ec8-4413-4f12-aa67-597524a328b4\") " pod="kuadrant-system/authorino-58cd955675-s5t54" Jul 02 19:10:56.740610 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.740575 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-ggnm9"] Jul 02 19:10:56.765576 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.765531 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ggnm9"] Jul 02 19:10:56.765750 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.765626 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ggnm9" Jul 02 19:10:56.809810 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.809730 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv9hq\" (UniqueName: \"kubernetes.io/projected/9239b437-a38d-4a1b-9c71-fd5471ab5427-kube-api-access-vv9hq\") pod \"authorino-75d556bd77-ggnm9\" (UID: \"9239b437-a38d-4a1b-9c71-fd5471ab5427\") " pod="kuadrant-system/authorino-75d556bd77-ggnm9" Jul 02 19:10:56.880359 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.880327 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s5t54" Jul 02 19:10:56.911247 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.911217 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv9hq\" (UniqueName: \"kubernetes.io/projected/9239b437-a38d-4a1b-9c71-fd5471ab5427-kube-api-access-vv9hq\") pod \"authorino-75d556bd77-ggnm9\" (UID: \"9239b437-a38d-4a1b-9c71-fd5471ab5427\") " pod="kuadrant-system/authorino-75d556bd77-ggnm9" Jul 02 19:10:56.920838 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:56.920807 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv9hq\" (UniqueName: \"kubernetes.io/projected/9239b437-a38d-4a1b-9c71-fd5471ab5427-kube-api-access-vv9hq\") pod \"authorino-75d556bd77-ggnm9\" (UID: \"9239b437-a38d-4a1b-9c71-fd5471ab5427\") " pod="kuadrant-system/authorino-75d556bd77-ggnm9" Jul 02 19:10:57.030041 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:57.030014 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s5t54"] Jul 02 19:10:57.032197 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:10:57.032166 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb7e1ec8_4413_4f12_aa67_597524a328b4.slice/crio-9ab85702166a6832a41ae50c772ad6c1bf84b811334376d1d997bdb22422230c WatchSource:0}: Error finding container 9ab85702166a6832a41ae50c772ad6c1bf84b811334376d1d997bdb22422230c: Status 404 returned error can't find the container with id 9ab85702166a6832a41ae50c772ad6c1bf84b811334376d1d997bdb22422230c Jul 02 19:10:57.075334 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:57.075257 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ggnm9" Jul 02 19:10:57.259993 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:57.259956 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ggnm9"] Jul 02 19:10:57.591793 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:57.591755 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ggnm9" event={"ID":"9239b437-a38d-4a1b-9c71-fd5471ab5427","Type":"ContainerStarted","Data":"0163ba9714450c34807d909884e327650731d236c14c6a711910555efd69653f"} Jul 02 19:10:57.593392 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:10:57.593347 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s5t54" event={"ID":"cb7e1ec8-4413-4f12-aa67-597524a328b4","Type":"ContainerStarted","Data":"9ab85702166a6832a41ae50c772ad6c1bf84b811334376d1d997bdb22422230c"} Jul 02 19:11:02.647354 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:02.647314 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" event={"ID":"80eddcda-1461-401a-9bb2-ba5e3f05f52d","Type":"ContainerStarted","Data":"59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72"} Jul 02 19:11:02.647824 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:02.647496 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:11:02.648992 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:02.648963 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ggnm9" event={"ID":"9239b437-a38d-4a1b-9c71-fd5471ab5427","Type":"ContainerStarted","Data":"49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb"} Jul 02 19:11:02.650338 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:02.650318 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s5t54" event={"ID":"cb7e1ec8-4413-4f12-aa67-597524a328b4","Type":"ContainerStarted","Data":"08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd"} Jul 02 19:11:02.671590 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:02.671531 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" podStartSLOduration=2.385967943 podStartE2EDuration="7.671489538s" totalImagesPullingTime="5.285521595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:02.668513618 +0000 UTC m=+499.959850518" watchObservedRunningTime="2026-07-02 19:11:02.671489538 +0000 UTC m=+499.962826438" Jul 02 19:11:02.691771 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:02.691719 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-ggnm9" podStartSLOduration=2.272880878 podStartE2EDuration="6.691701538s" totalImagesPullingTime="4.41882066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:02.689605215 +0000 UTC m=+499.980942116" watchObservedRunningTime="2026-07-02 19:11:02.691701538 +0000 UTC m=+499.983038437" Jul 02 19:11:02.724325 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:02.724268 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-s5t54" podStartSLOduration=2.058671343 podStartE2EDuration="6.724251278s" totalImagesPullingTime="4.665579935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:02.722848156 +0000 UTC m=+500.014185057" watchObservedRunningTime="2026-07-02 19:11:02.724251278 +0000 UTC m=+500.015588178" Jul 02 19:11:02.775061 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:02.775024 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s5t54"] Jul 02 19:11:04.661392 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:04.661329 2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-s5t54" podUID="cb7e1ec8-4413-4f12-aa67-597524a328b4" containerName="authorino" containerID="cri-o://08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd" gracePeriod=30 Jul 02 19:11:04.926796 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:04.926730 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s5t54" Jul 02 19:11:04.986642 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:04.986600 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5bk7\" (UniqueName: \"kubernetes.io/projected/cb7e1ec8-4413-4f12-aa67-597524a328b4-kube-api-access-l5bk7\") pod \"cb7e1ec8-4413-4f12-aa67-597524a328b4\" (UID: \"cb7e1ec8-4413-4f12-aa67-597524a328b4\") " Jul 02 19:11:04.988737 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:04.988712 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb7e1ec8-4413-4f12-aa67-597524a328b4-kube-api-access-l5bk7" (OuterVolumeSpecName: "kube-api-access-l5bk7") pod "cb7e1ec8-4413-4f12-aa67-597524a328b4" (UID: "cb7e1ec8-4413-4f12-aa67-597524a328b4"). InnerVolumeSpecName "kube-api-access-l5bk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:11:05.087548 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:05.087513 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5bk7\" (UniqueName: \"kubernetes.io/projected/cb7e1ec8-4413-4f12-aa67-597524a328b4-kube-api-access-l5bk7\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:11:05.668015 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:05.667975 2571 generic.go:356] "Generic (PLEG): container finished" podID="cb7e1ec8-4413-4f12-aa67-597524a328b4" containerID="08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd" exitCode=0 Jul 02 19:11:05.668015 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:05.668017 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s5t54" Jul 02 19:11:05.668558 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:05.668054 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s5t54" event={"ID":"cb7e1ec8-4413-4f12-aa67-597524a328b4","Type":"ContainerDied","Data":"08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd"} Jul 02 19:11:05.668558 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:05.668114 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s5t54" event={"ID":"cb7e1ec8-4413-4f12-aa67-597524a328b4","Type":"ContainerDied","Data":"9ab85702166a6832a41ae50c772ad6c1bf84b811334376d1d997bdb22422230c"} Jul 02 19:11:05.668558 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:05.668135 2571 scope.go:117] "RemoveContainer" containerID="08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd" Jul 02 19:11:05.677167 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:05.677142 2571 scope.go:117] "RemoveContainer" containerID="08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd" Jul 02 19:11:05.677443 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:11:05.677423 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd\": container with ID starting with 08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd not found: ID does not exist" containerID="08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd" Jul 02 19:11:05.677493 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:05.677454 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd"} err="failed to get container status \"08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd\": rpc error: code = NotFound desc = could not find container \"08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd\": container with ID starting with 08762e085e6279b6ad8eb9952e4662e3a70168dd0cd7dbd0e4f35bba76889fcd not found: ID does not exist" Jul 02 19:11:05.688550 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:05.688524 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s5t54"] Jul 02 19:11:05.699986 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:05.699959 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-s5t54"] Jul 02 19:11:07.237689 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:07.237656 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb7e1ec8-4413-4f12-aa67-597524a328b4" path="/var/lib/kubelet/pods/cb7e1ec8-4413-4f12-aa67-597524a328b4/volumes" Jul 02 19:11:10.495994 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:10.495957 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sz8w8"] Jul 02 19:11:10.496437 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:10.496210 2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" podUID="80eddcda-1461-401a-9bb2-ba5e3f05f52d" containerName="limitador" containerID="cri-o://59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72" gracePeriod=30 Jul 02 19:11:10.496910 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:10.496799 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:11:11.057687 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.057630 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:11:11.142029 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.141999 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/80eddcda-1461-401a-9bb2-ba5e3f05f52d-config-file\") pod \"80eddcda-1461-401a-9bb2-ba5e3f05f52d\" (UID: \"80eddcda-1461-401a-9bb2-ba5e3f05f52d\") " Jul 02 19:11:11.142252 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.142036 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rr5pb\" (UniqueName: \"kubernetes.io/projected/80eddcda-1461-401a-9bb2-ba5e3f05f52d-kube-api-access-rr5pb\") pod \"80eddcda-1461-401a-9bb2-ba5e3f05f52d\" (UID: \"80eddcda-1461-401a-9bb2-ba5e3f05f52d\") " Jul 02 19:11:11.142471 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.142442 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80eddcda-1461-401a-9bb2-ba5e3f05f52d-config-file" (OuterVolumeSpecName: "config-file") pod "80eddcda-1461-401a-9bb2-ba5e3f05f52d" (UID: "80eddcda-1461-401a-9bb2-ba5e3f05f52d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:11:11.144214 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.144191 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80eddcda-1461-401a-9bb2-ba5e3f05f52d-kube-api-access-rr5pb" (OuterVolumeSpecName: "kube-api-access-rr5pb") pod "80eddcda-1461-401a-9bb2-ba5e3f05f52d" (UID: "80eddcda-1461-401a-9bb2-ba5e3f05f52d"). InnerVolumeSpecName "kube-api-access-rr5pb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:11:11.243286 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.243258 2571 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/80eddcda-1461-401a-9bb2-ba5e3f05f52d-config-file\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:11:11.243286 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.243283 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr5pb\" (UniqueName: \"kubernetes.io/projected/80eddcda-1461-401a-9bb2-ba5e3f05f52d-kube-api-access-rr5pb\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:11:11.699374 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.699336 2571 generic.go:356] "Generic (PLEG): container finished" podID="80eddcda-1461-401a-9bb2-ba5e3f05f52d" containerID="59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72" exitCode=0 Jul 02 19:11:11.699799 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.699382 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" Jul 02 19:11:11.699799 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.699416 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" event={"ID":"80eddcda-1461-401a-9bb2-ba5e3f05f52d","Type":"ContainerDied","Data":"59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72"} Jul 02 19:11:11.699799 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.699469 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sz8w8" event={"ID":"80eddcda-1461-401a-9bb2-ba5e3f05f52d","Type":"ContainerDied","Data":"a82e279b8e39798d0b24be54432d116eb8be2ab1babe7e9c7dd5284e3055c28d"} Jul 02 19:11:11.699799 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.699492 2571 scope.go:117] "RemoveContainer" containerID="59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72" Jul 02 19:11:11.708281 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.708265 2571 scope.go:117] "RemoveContainer" containerID="59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72" Jul 02 19:11:11.708549 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:11:11.708529 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72\": container with ID starting with 59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72 not found: ID does not exist" containerID="59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72" Jul 02 19:11:11.708602 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.708557 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72"} err="failed to get container status \"59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72\": rpc error: code = NotFound desc = could not find container \"59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72\": container with ID starting with 59258266f1f03a0cb083d6d714c8fd570a55622db163279d9c28d3ee5aabde72 not found: ID does not exist" Jul 02 19:11:11.717925 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.717901 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sz8w8"] Jul 02 19:11:11.726034 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:11.726012 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sz8w8"] Jul 02 19:11:12.260272 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.260239 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-xzbfb"] Jul 02 19:11:12.260781 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.260763 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80eddcda-1461-401a-9bb2-ba5e3f05f52d" containerName="limitador" Jul 02 19:11:12.260834 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.260783 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="80eddcda-1461-401a-9bb2-ba5e3f05f52d" containerName="limitador" Jul 02 19:11:12.260834 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.260804 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb7e1ec8-4413-4f12-aa67-597524a328b4" containerName="authorino" Jul 02 19:11:12.260834 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.260809 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb7e1ec8-4413-4f12-aa67-597524a328b4" containerName="authorino" Jul 02 19:11:12.260924 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.260881 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="80eddcda-1461-401a-9bb2-ba5e3f05f52d" containerName="limitador" Jul 02 19:11:12.260924 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.260891 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb7e1ec8-4413-4f12-aa67-597524a328b4" containerName="authorino" Jul 02 19:11:12.263667 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.263651 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-xzbfb" Jul 02 19:11:12.268055 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.268036 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 02 19:11:12.268166 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.268036 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5jhfg\"" Jul 02 19:11:12.276762 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.276739 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-xzbfb"] Jul 02 19:11:12.354828 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.354789 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/570c0902-be34-4482-b33f-71ecf681f742-data\") pod \"postgres-69fb4594cf-xzbfb\" (UID: \"570c0902-be34-4482-b33f-71ecf681f742\") " pod="opendatahub/postgres-69fb4594cf-xzbfb" Jul 02 19:11:12.355006 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.354927 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwmt4\" (UniqueName: \"kubernetes.io/projected/570c0902-be34-4482-b33f-71ecf681f742-kube-api-access-lwmt4\") pod \"postgres-69fb4594cf-xzbfb\" (UID: \"570c0902-be34-4482-b33f-71ecf681f742\") " pod="opendatahub/postgres-69fb4594cf-xzbfb" Jul 02 19:11:12.456471 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.456437 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/570c0902-be34-4482-b33f-71ecf681f742-data\") pod \"postgres-69fb4594cf-xzbfb\" (UID: \"570c0902-be34-4482-b33f-71ecf681f742\") " pod="opendatahub/postgres-69fb4594cf-xzbfb" Jul 02 19:11:12.456643 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.456500 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwmt4\" (UniqueName: \"kubernetes.io/projected/570c0902-be34-4482-b33f-71ecf681f742-kube-api-access-lwmt4\") pod \"postgres-69fb4594cf-xzbfb\" (UID: \"570c0902-be34-4482-b33f-71ecf681f742\") " pod="opendatahub/postgres-69fb4594cf-xzbfb" Jul 02 19:11:12.456813 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.456794 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/570c0902-be34-4482-b33f-71ecf681f742-data\") pod \"postgres-69fb4594cf-xzbfb\" (UID: \"570c0902-be34-4482-b33f-71ecf681f742\") " pod="opendatahub/postgres-69fb4594cf-xzbfb" Jul 02 19:11:12.466638 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.466607 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwmt4\" (UniqueName: \"kubernetes.io/projected/570c0902-be34-4482-b33f-71ecf681f742-kube-api-access-lwmt4\") pod \"postgres-69fb4594cf-xzbfb\" (UID: \"570c0902-be34-4482-b33f-71ecf681f742\") " pod="opendatahub/postgres-69fb4594cf-xzbfb" Jul 02 19:11:12.574096 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.574035 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-xzbfb" Jul 02 19:11:12.712711 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:12.712690 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-xzbfb"] Jul 02 19:11:12.714388 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:11:12.714362 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570c0902_be34_4482_b33f_71ecf681f742.slice/crio-0afbe6aa1d7b000b54e9b672b093ec31958a8f5936f152651e99626a0012eba0 WatchSource:0}: Error finding container 0afbe6aa1d7b000b54e9b672b093ec31958a8f5936f152651e99626a0012eba0: Status 404 returned error can't find the container with id 0afbe6aa1d7b000b54e9b672b093ec31958a8f5936f152651e99626a0012eba0 Jul 02 19:11:13.237994 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:13.237953 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80eddcda-1461-401a-9bb2-ba5e3f05f52d" path="/var/lib/kubelet/pods/80eddcda-1461-401a-9bb2-ba5e3f05f52d/volumes" Jul 02 19:11:13.711791 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:13.711750 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-xzbfb" event={"ID":"570c0902-be34-4482-b33f-71ecf681f742","Type":"ContainerStarted","Data":"0afbe6aa1d7b000b54e9b672b093ec31958a8f5936f152651e99626a0012eba0"} Jul 02 19:11:18.749286 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:18.749246 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-xzbfb" event={"ID":"570c0902-be34-4482-b33f-71ecf681f742","Type":"ContainerStarted","Data":"7f438227c0f6b410d0723033599c64ea062f74e1c30226c88a6ab7d6bb67d09d"} Jul 02 19:11:18.749675 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:18.749347 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-xzbfb" Jul 02 19:11:18.772035 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:18.771990 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-xzbfb" podStartSLOduration=0.983252042 podStartE2EDuration="6.771974056s" totalImagesPullingTime="5.788722014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:18.770759918 +0000 UTC m=+516.062096818" watchObservedRunningTime="2026-07-02 19:11:18.771974056 +0000 UTC m=+516.063310956" Jul 02 19:11:24.784215 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:24.784185 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-xzbfb" Jul 02 19:11:25.340209 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.340176 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w25vr"] Jul 02 19:11:25.346821 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.346795 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w25vr" Jul 02 19:11:25.356063 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.356032 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w25vr"] Jul 02 19:11:25.477293 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.477258 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8qd5\" (UniqueName: \"kubernetes.io/projected/85b552d6-144d-452e-a0c3-4e2f0fb4fcb6-kube-api-access-k8qd5\") pod \"authorino-5d4b67c446-w25vr\" (UID: \"85b552d6-144d-452e-a0c3-4e2f0fb4fcb6\") " pod="kuadrant-system/authorino-5d4b67c446-w25vr" Jul 02 19:11:25.573633 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.573596 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w25vr"] Jul 02 19:11:25.573927 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:11:25.573904 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-k8qd5], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-w25vr" podUID="85b552d6-144d-452e-a0c3-4e2f0fb4fcb6" Jul 02 19:11:25.578518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.578484 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8qd5\" (UniqueName: \"kubernetes.io/projected/85b552d6-144d-452e-a0c3-4e2f0fb4fcb6-kube-api-access-k8qd5\") pod \"authorino-5d4b67c446-w25vr\" (UID: \"85b552d6-144d-452e-a0c3-4e2f0fb4fcb6\") " pod="kuadrant-system/authorino-5d4b67c446-w25vr" Jul 02 19:11:25.587108 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.587065 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8qd5\" (UniqueName: \"kubernetes.io/projected/85b552d6-144d-452e-a0c3-4e2f0fb4fcb6-kube-api-access-k8qd5\") pod \"authorino-5d4b67c446-w25vr\" (UID: \"85b552d6-144d-452e-a0c3-4e2f0fb4fcb6\") " pod="kuadrant-system/authorino-5d4b67c446-w25vr" Jul 02 19:11:25.602262 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.602185 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-795645855-4c65p"] Jul 02 19:11:25.605893 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.605869 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-795645855-4c65p" Jul 02 19:11:25.616964 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.616936 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-795645855-4c65p"] Jul 02 19:11:25.646426 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.646392 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-795645855-4c65p"] Jul 02 19:11:25.646807 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:11:25.646771 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hqw5d], unattached volumes=[], failed to process volumes=[kube-api-access-hqw5d]: context canceled" pod="kuadrant-system/authorino-795645855-4c65p" podUID="3f519f59-6844-4e8b-a36c-5095a9357e5c" Jul 02 19:11:25.684429 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.684392 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-89b864fc5-78k8q"] Jul 02 19:11:25.687997 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.687980 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-89b864fc5-78k8q" Jul 02 19:11:25.690977 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.690953 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 02 19:11:25.696563 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.696540 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-89b864fc5-78k8q"] Jul 02 19:11:25.779864 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.779824 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-tls-cert\") pod \"authorino-89b864fc5-78k8q\" (UID: \"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6\") " pod="kuadrant-system/authorino-89b864fc5-78k8q" Jul 02 19:11:25.780059 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.779876 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqw5d\" (UniqueName: \"kubernetes.io/projected/3f519f59-6844-4e8b-a36c-5095a9357e5c-kube-api-access-hqw5d\") pod \"authorino-795645855-4c65p\" (UID: \"3f519f59-6844-4e8b-a36c-5095a9357e5c\") " pod="kuadrant-system/authorino-795645855-4c65p" Jul 02 19:11:25.780059 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.779906 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq4ls\" (UniqueName: \"kubernetes.io/projected/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-kube-api-access-rq4ls\") pod \"authorino-89b864fc5-78k8q\" (UID: \"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6\") " pod="kuadrant-system/authorino-89b864fc5-78k8q" Jul 02 19:11:25.791567 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.791543 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-795645855-4c65p" Jul 02 19:11:25.792032 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.791543 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w25vr" Jul 02 19:11:25.798639 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.798616 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-795645855-4c65p" Jul 02 19:11:25.801739 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.801722 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w25vr" Jul 02 19:11:25.881124 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.881003 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8qd5\" (UniqueName: \"kubernetes.io/projected/85b552d6-144d-452e-a0c3-4e2f0fb4fcb6-kube-api-access-k8qd5\") pod \"85b552d6-144d-452e-a0c3-4e2f0fb4fcb6\" (UID: \"85b552d6-144d-452e-a0c3-4e2f0fb4fcb6\") " Jul 02 19:11:25.881304 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.881174 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqw5d\" (UniqueName: \"kubernetes.io/projected/3f519f59-6844-4e8b-a36c-5095a9357e5c-kube-api-access-hqw5d\") pod \"authorino-795645855-4c65p\" (UID: \"3f519f59-6844-4e8b-a36c-5095a9357e5c\") " pod="kuadrant-system/authorino-795645855-4c65p" Jul 02 19:11:25.881304 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.881212 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4ls\" (UniqueName: \"kubernetes.io/projected/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-kube-api-access-rq4ls\") pod \"authorino-89b864fc5-78k8q\" (UID: \"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6\") " pod="kuadrant-system/authorino-89b864fc5-78k8q" Jul 02 19:11:25.881304 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.881295 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-tls-cert\") pod \"authorino-89b864fc5-78k8q\" (UID: \"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6\") " pod="kuadrant-system/authorino-89b864fc5-78k8q" Jul 02 19:11:25.883301 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.883274 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85b552d6-144d-452e-a0c3-4e2f0fb4fcb6-kube-api-access-k8qd5" (OuterVolumeSpecName: "kube-api-access-k8qd5") pod "85b552d6-144d-452e-a0c3-4e2f0fb4fcb6" (UID: "85b552d6-144d-452e-a0c3-4e2f0fb4fcb6"). InnerVolumeSpecName "kube-api-access-k8qd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:11:25.883561 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.883543 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-tls-cert\") pod \"authorino-89b864fc5-78k8q\" (UID: \"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6\") " pod="kuadrant-system/authorino-89b864fc5-78k8q" Jul 02 19:11:25.892551 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.892530 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq4ls\" (UniqueName: \"kubernetes.io/projected/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-kube-api-access-rq4ls\") pod \"authorino-89b864fc5-78k8q\" (UID: \"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6\") " pod="kuadrant-system/authorino-89b864fc5-78k8q" Jul 02 19:11:25.892672 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.892649 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqw5d\" (UniqueName: \"kubernetes.io/projected/3f519f59-6844-4e8b-a36c-5095a9357e5c-kube-api-access-hqw5d\") pod \"authorino-795645855-4c65p\" (UID: \"3f519f59-6844-4e8b-a36c-5095a9357e5c\") " pod="kuadrant-system/authorino-795645855-4c65p" Jul 02 19:11:25.982470 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.982438 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqw5d\" (UniqueName: \"kubernetes.io/projected/3f519f59-6844-4e8b-a36c-5095a9357e5c-kube-api-access-hqw5d\") pod \"3f519f59-6844-4e8b-a36c-5095a9357e5c\" (UID: \"3f519f59-6844-4e8b-a36c-5095a9357e5c\") " Jul 02 19:11:25.982649 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.982636 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8qd5\" (UniqueName: \"kubernetes.io/projected/85b552d6-144d-452e-a0c3-4e2f0fb4fcb6-kube-api-access-k8qd5\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:11:25.984474 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.984450 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f519f59-6844-4e8b-a36c-5095a9357e5c-kube-api-access-hqw5d" (OuterVolumeSpecName: "kube-api-access-hqw5d") pod "3f519f59-6844-4e8b-a36c-5095a9357e5c" (UID: "3f519f59-6844-4e8b-a36c-5095a9357e5c"). InnerVolumeSpecName "kube-api-access-hqw5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:11:25.997433 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:25.997411 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-89b864fc5-78k8q" Jul 02 19:11:26.083638 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.083601 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqw5d\" (UniqueName: \"kubernetes.io/projected/3f519f59-6844-4e8b-a36c-5095a9357e5c-kube-api-access-hqw5d\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:11:26.131985 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.131958 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-89b864fc5-78k8q"] Jul 02 19:11:26.133806 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:11:26.133776 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d39c7a9_1f3b_43a2_ae78_d55a3c176be6.slice/crio-36433246d629b92eca55f04036424b0f168c4907da346767d980853286d4bce7 WatchSource:0}: Error finding container 36433246d629b92eca55f04036424b0f168c4907da346767d980853286d4bce7: Status 404 returned error can't find the container with id 36433246d629b92eca55f04036424b0f168c4907da346767d980853286d4bce7 Jul 02 19:11:26.798421 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.798384 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-89b864fc5-78k8q" event={"ID":"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6","Type":"ContainerStarted","Data":"6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18"} Jul 02 19:11:26.798421 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.798427 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-89b864fc5-78k8q" event={"ID":"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6","Type":"ContainerStarted","Data":"36433246d629b92eca55f04036424b0f168c4907da346767d980853286d4bce7"} Jul 02 19:11:26.798942 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.798434 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w25vr" Jul 02 19:11:26.798942 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.798592 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-795645855-4c65p" Jul 02 19:11:26.821011 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.820960 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-89b864fc5-78k8q" podStartSLOduration=1.5056081479999999 podStartE2EDuration="1.820944198s" totalImagesPullingTime="315.33605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:26.817440365 +0000 UTC m=+524.108777265" watchObservedRunningTime="2026-07-02 19:11:26.820944198 +0000 UTC m=+524.112281098" Jul 02 19:11:26.857422 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.857383 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w25vr"] Jul 02 19:11:26.866286 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.866254 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ggnm9"] Jul 02 19:11:26.866454 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.866432 2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-ggnm9" podUID="9239b437-a38d-4a1b-9c71-fd5471ab5427" containerName="authorino" containerID="cri-o://49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb" gracePeriod=30 Jul 02 19:11:26.869944 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.869921 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w25vr"] Jul 02 19:11:26.905572 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.905537 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-795645855-4c65p"] Jul 02 19:11:26.914293 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:26.914262 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-795645855-4c65p"] Jul 02 19:11:27.128007 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.127980 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ggnm9" Jul 02 19:11:27.238790 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.238754 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f519f59-6844-4e8b-a36c-5095a9357e5c" path="/var/lib/kubelet/pods/3f519f59-6844-4e8b-a36c-5095a9357e5c/volumes" Jul 02 19:11:27.239151 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.239133 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85b552d6-144d-452e-a0c3-4e2f0fb4fcb6" path="/var/lib/kubelet/pods/85b552d6-144d-452e-a0c3-4e2f0fb4fcb6/volumes" Jul 02 19:11:27.293712 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.293681 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv9hq\" (UniqueName: \"kubernetes.io/projected/9239b437-a38d-4a1b-9c71-fd5471ab5427-kube-api-access-vv9hq\") pod \"9239b437-a38d-4a1b-9c71-fd5471ab5427\" (UID: \"9239b437-a38d-4a1b-9c71-fd5471ab5427\") " Jul 02 19:11:27.295681 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.295653 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9239b437-a38d-4a1b-9c71-fd5471ab5427-kube-api-access-vv9hq" (OuterVolumeSpecName: "kube-api-access-vv9hq") pod "9239b437-a38d-4a1b-9c71-fd5471ab5427" (UID: "9239b437-a38d-4a1b-9c71-fd5471ab5427"). InnerVolumeSpecName "kube-api-access-vv9hq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:11:27.394798 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.394720 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv9hq\" (UniqueName: \"kubernetes.io/projected/9239b437-a38d-4a1b-9c71-fd5471ab5427-kube-api-access-vv9hq\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:11:27.804742 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.804704 2571 generic.go:356] "Generic (PLEG): container finished" podID="9239b437-a38d-4a1b-9c71-fd5471ab5427" containerID="49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb" exitCode=0 Jul 02 19:11:27.805196 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.804750 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ggnm9" Jul 02 19:11:27.805196 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.804789 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ggnm9" event={"ID":"9239b437-a38d-4a1b-9c71-fd5471ab5427","Type":"ContainerDied","Data":"49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb"} Jul 02 19:11:27.805196 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.804831 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ggnm9" event={"ID":"9239b437-a38d-4a1b-9c71-fd5471ab5427","Type":"ContainerDied","Data":"0163ba9714450c34807d909884e327650731d236c14c6a711910555efd69653f"} Jul 02 19:11:27.805196 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.804850 2571 scope.go:117] "RemoveContainer" containerID="49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb" Jul 02 19:11:27.814039 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.814021 2571 scope.go:117] "RemoveContainer" containerID="49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb" Jul 02 19:11:27.814350 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:11:27.814332 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb\": container with ID starting with 49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb not found: ID does not exist" containerID="49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb" Jul 02 19:11:27.814413 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.814359 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb"} err="failed to get container status \"49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb\": rpc error: code = NotFound desc = could not find container \"49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb\": container with ID starting with 49ca2a6f8e8e8b791aef73ebdd3b9779a65fb4acb83620a723477c7e49563beb not found: ID does not exist" Jul 02 19:11:27.841225 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.841194 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ggnm9"] Jul 02 19:11:27.843819 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:27.843798 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ggnm9"] Jul 02 19:11:29.237644 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:29.237611 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9239b437-a38d-4a1b-9c71-fd5471ab5427" path="/var/lib/kubelet/pods/9239b437-a38d-4a1b-9c71-fd5471ab5427/volumes" Jul 02 19:11:31.668085 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.668033 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-599fb795f-82xsh"] Jul 02 19:11:31.669515 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.669486 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9239b437-a38d-4a1b-9c71-fd5471ab5427" containerName="authorino" Jul 02 19:11:31.669706 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.669695 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9239b437-a38d-4a1b-9c71-fd5471ab5427" containerName="authorino" Jul 02 19:11:31.670023 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.670003 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9239b437-a38d-4a1b-9c71-fd5471ab5427" containerName="authorino" Jul 02 19:11:31.676807 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.676779 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:31.684151 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.684124 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 02 19:11:31.684279 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.684124 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6xt7d\"" Jul 02 19:11:31.684279 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.684136 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 02 19:11:31.704742 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.704712 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-599fb795f-82xsh"] Jul 02 19:11:31.836564 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.836528 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9bwx\" (UniqueName: \"kubernetes.io/projected/43dff504-70c6-4068-b83e-f3f7db76662c-kube-api-access-s9bwx\") pod \"maas-controller-599fb795f-82xsh\" (UID: \"43dff504-70c6-4068-b83e-f3f7db76662c\") " pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:31.836789 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.836701 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43dff504-70c6-4068-b83e-f3f7db76662c-webhook-cert\") pod \"maas-controller-599fb795f-82xsh\" (UID: \"43dff504-70c6-4068-b83e-f3f7db76662c\") " pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:31.850508 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.850460 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85fc68f447-zkm6j"] Jul 02 19:11:31.854466 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.854449 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:11:31.872614 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.872584 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85fc68f447-zkm6j"] Jul 02 19:11:31.937789 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.937698 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9bwx\" (UniqueName: \"kubernetes.io/projected/43dff504-70c6-4068-b83e-f3f7db76662c-kube-api-access-s9bwx\") pod \"maas-controller-599fb795f-82xsh\" (UID: \"43dff504-70c6-4068-b83e-f3f7db76662c\") " pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:31.937950 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.937820 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43dff504-70c6-4068-b83e-f3f7db76662c-webhook-cert\") pod \"maas-controller-599fb795f-82xsh\" (UID: \"43dff504-70c6-4068-b83e-f3f7db76662c\") " pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:31.940185 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.940166 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43dff504-70c6-4068-b83e-f3f7db76662c-webhook-cert\") pod \"maas-controller-599fb795f-82xsh\" (UID: \"43dff504-70c6-4068-b83e-f3f7db76662c\") " pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:31.960117 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.960083 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9bwx\" (UniqueName: \"kubernetes.io/projected/43dff504-70c6-4068-b83e-f3f7db76662c-kube-api-access-s9bwx\") pod \"maas-controller-599fb795f-82xsh\" (UID: \"43dff504-70c6-4068-b83e-f3f7db76662c\") " pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:31.986988 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:31.986955 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:32.039035 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:32.038963 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q6dc\" (UniqueName: \"kubernetes.io/projected/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-kube-api-access-2q6dc\") pod \"maas-controller-85fc68f447-zkm6j\" (UID: \"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7\") " pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:11:32.039035 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:32.039039 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-webhook-cert\") pod \"maas-controller-85fc68f447-zkm6j\" (UID: \"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7\") " pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:11:32.140649 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:32.140618 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q6dc\" (UniqueName: \"kubernetes.io/projected/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-kube-api-access-2q6dc\") pod \"maas-controller-85fc68f447-zkm6j\" (UID: \"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7\") " pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:11:32.140771 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:32.140702 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-webhook-cert\") pod \"maas-controller-85fc68f447-zkm6j\" (UID: \"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7\") " pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:11:32.141206 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:32.141187 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-599fb795f-82xsh"] Jul 02 19:11:32.142134 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:11:32.142102 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43dff504_70c6_4068_b83e_f3f7db76662c.slice/crio-f6aff71a4f58be27fb3923f9d9d46e92cce65dea3be5555f5991f64726594a1b WatchSource:0}: Error finding container f6aff71a4f58be27fb3923f9d9d46e92cce65dea3be5555f5991f64726594a1b: Status 404 returned error can't find the container with id f6aff71a4f58be27fb3923f9d9d46e92cce65dea3be5555f5991f64726594a1b Jul 02 19:11:32.143273 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:32.143252 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-webhook-cert\") pod \"maas-controller-85fc68f447-zkm6j\" (UID: \"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7\") " pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:11:32.154451 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:32.154426 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q6dc\" (UniqueName: \"kubernetes.io/projected/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-kube-api-access-2q6dc\") pod \"maas-controller-85fc68f447-zkm6j\" (UID: \"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7\") " pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:11:32.165223 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:32.165193 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:11:32.320187 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:32.320162 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85fc68f447-zkm6j"] Jul 02 19:11:32.321390 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:11:32.321365 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b1875bc_fb16_4dd4_8d1f_c9848049c3c7.slice/crio-2013eaf041d72bd7df2a7b8b2f55a2d6e70cf3d9c6a50897161f1e793fc24d47 WatchSource:0}: Error finding container 2013eaf041d72bd7df2a7b8b2f55a2d6e70cf3d9c6a50897161f1e793fc24d47: Status 404 returned error can't find the container with id 2013eaf041d72bd7df2a7b8b2f55a2d6e70cf3d9c6a50897161f1e793fc24d47 Jul 02 19:11:32.839309 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:32.839267 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85fc68f447-zkm6j" event={"ID":"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7","Type":"ContainerStarted","Data":"2013eaf041d72bd7df2a7b8b2f55a2d6e70cf3d9c6a50897161f1e793fc24d47"} Jul 02 19:11:32.842962 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:32.842930 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-599fb795f-82xsh" event={"ID":"43dff504-70c6-4068-b83e-f3f7db76662c","Type":"ContainerStarted","Data":"f6aff71a4f58be27fb3923f9d9d46e92cce65dea3be5555f5991f64726594a1b"} Jul 02 19:11:34.857688 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:34.857645 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85fc68f447-zkm6j" event={"ID":"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7","Type":"ContainerStarted","Data":"94b6348aca24b6ee0f084456630b1afcc301a615d1862451a2798da3be0fdacc"} Jul 02 19:11:34.858190 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:34.857800 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:11:34.859476 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:34.859452 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-599fb795f-82xsh" event={"ID":"43dff504-70c6-4068-b83e-f3f7db76662c","Type":"ContainerStarted","Data":"6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7"} Jul 02 19:11:34.859582 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:34.859535 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:34.928796 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:34.928733 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85fc68f447-zkm6j" podStartSLOduration=1.982400702 podStartE2EDuration="3.928714405s" totalImagesPullingTime="1.946313703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:34.895153898 +0000 UTC m=+532.186490800" watchObservedRunningTime="2026-07-02 19:11:34.928714405 +0000 UTC m=+532.220051305" Jul 02 19:11:34.929422 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:34.929388 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-599fb795f-82xsh" podStartSLOduration=1.81474411 podStartE2EDuration="3.929378867s" totalImagesPullingTime="2.114634757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:34.928951022 +0000 UTC m=+532.220287947" watchObservedRunningTime="2026-07-02 19:11:34.929378867 +0000 UTC m=+532.220715766" Jul 02 19:11:45.870448 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:45.870421 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:45.870904 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:45.870740 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:11:45.968399 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:45.968364 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-599fb795f-82xsh"] Jul 02 19:11:45.968965 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:45.968908 2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-599fb795f-82xsh" podUID="43dff504-70c6-4068-b83e-f3f7db76662c" containerName="manager" containerID="cri-o://6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7" gracePeriod=30 Jul 02 19:11:46.223580 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.223552 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:46.264108 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.264062 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9bwx\" (UniqueName: \"kubernetes.io/projected/43dff504-70c6-4068-b83e-f3f7db76662c-kube-api-access-s9bwx\") pod \"43dff504-70c6-4068-b83e-f3f7db76662c\" (UID: \"43dff504-70c6-4068-b83e-f3f7db76662c\") " Jul 02 19:11:46.264267 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.264117 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43dff504-70c6-4068-b83e-f3f7db76662c-webhook-cert\") pod \"43dff504-70c6-4068-b83e-f3f7db76662c\" (UID: \"43dff504-70c6-4068-b83e-f3f7db76662c\") " Jul 02 19:11:46.266326 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.266296 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43dff504-70c6-4068-b83e-f3f7db76662c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "43dff504-70c6-4068-b83e-f3f7db76662c" (UID: "43dff504-70c6-4068-b83e-f3f7db76662c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:11:46.266501 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.266408 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43dff504-70c6-4068-b83e-f3f7db76662c-kube-api-access-s9bwx" (OuterVolumeSpecName: "kube-api-access-s9bwx") pod "43dff504-70c6-4068-b83e-f3f7db76662c" (UID: "43dff504-70c6-4068-b83e-f3f7db76662c"). InnerVolumeSpecName "kube-api-access-s9bwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:11:46.365246 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.365210 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9bwx\" (UniqueName: \"kubernetes.io/projected/43dff504-70c6-4068-b83e-f3f7db76662c-kube-api-access-s9bwx\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:11:46.365246 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.365240 2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43dff504-70c6-4068-b83e-f3f7db76662c-webhook-cert\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:11:46.927212 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.927172 2571 generic.go:356] "Generic (PLEG): container finished" podID="43dff504-70c6-4068-b83e-f3f7db76662c" containerID="6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7" exitCode=0 Jul 02 19:11:46.927687 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.927216 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-599fb795f-82xsh" event={"ID":"43dff504-70c6-4068-b83e-f3f7db76662c","Type":"ContainerDied","Data":"6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7"} Jul 02 19:11:46.927687 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.927262 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-599fb795f-82xsh" event={"ID":"43dff504-70c6-4068-b83e-f3f7db76662c","Type":"ContainerDied","Data":"f6aff71a4f58be27fb3923f9d9d46e92cce65dea3be5555f5991f64726594a1b"} Jul 02 19:11:46.927687 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.927288 2571 scope.go:117] "RemoveContainer" containerID="6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7" Jul 02 19:11:46.927687 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.927219 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-599fb795f-82xsh" Jul 02 19:11:46.936880 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.936859 2571 scope.go:117] "RemoveContainer" containerID="6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7" Jul 02 19:11:46.937201 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:11:46.937184 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7\": container with ID starting with 6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7 not found: ID does not exist" containerID="6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7" Jul 02 19:11:46.937275 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.937209 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7"} err="failed to get container status \"6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7\": rpc error: code = NotFound desc = could not find container \"6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7\": container with ID starting with 6a37904e43c2b211532374174d6825d487e71951dc3df8130f892a58df8f7bd7 not found: ID does not exist" Jul 02 19:11:46.962249 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.962218 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-599fb795f-82xsh"] Jul 02 19:11:46.968210 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:46.968185 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-599fb795f-82xsh"] Jul 02 19:11:47.238448 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:11:47.238365 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43dff504-70c6-4068-b83e-f3f7db76662c" path="/var/lib/kubelet/pods/43dff504-70c6-4068-b83e-f3f7db76662c/volumes" Jul 02 19:12:07.680064 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.680029 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-9dcfd9c48-lgrqf"] Jul 02 19:12:07.680471 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.680460 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43dff504-70c6-4068-b83e-f3f7db76662c" containerName="manager" Jul 02 19:12:07.680471 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.680471 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="43dff504-70c6-4068-b83e-f3f7db76662c" containerName="manager" Jul 02 19:12:07.680544 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.680538 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="43dff504-70c6-4068-b83e-f3f7db76662c" containerName="manager" Jul 02 19:12:07.683558 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.683541 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:07.687152 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.687128 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 02 19:12:07.687378 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.687128 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-hxm9f\"" Jul 02 19:12:07.698809 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.698782 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-9dcfd9c48-lgrqf"] Jul 02 19:12:07.761912 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.761875 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/477b13c6-d88e-4e12-8fdc-ee7395f4c179-openshift-service-ca\") pod \"maas-api-9dcfd9c48-lgrqf\" (UID: \"477b13c6-d88e-4e12-8fdc-ee7395f4c179\") " pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:07.761912 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.761911 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgsr6\" (UniqueName: \"kubernetes.io/projected/477b13c6-d88e-4e12-8fdc-ee7395f4c179-kube-api-access-pgsr6\") pod \"maas-api-9dcfd9c48-lgrqf\" (UID: \"477b13c6-d88e-4e12-8fdc-ee7395f4c179\") " pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:07.762142 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.761950 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/477b13c6-d88e-4e12-8fdc-ee7395f4c179-maas-api-tls\") pod \"maas-api-9dcfd9c48-lgrqf\" (UID: \"477b13c6-d88e-4e12-8fdc-ee7395f4c179\") " pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:07.863433 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.863397 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/477b13c6-d88e-4e12-8fdc-ee7395f4c179-maas-api-tls\") pod \"maas-api-9dcfd9c48-lgrqf\" (UID: \"477b13c6-d88e-4e12-8fdc-ee7395f4c179\") " pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:07.863650 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.863519 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/477b13c6-d88e-4e12-8fdc-ee7395f4c179-openshift-service-ca\") pod \"maas-api-9dcfd9c48-lgrqf\" (UID: \"477b13c6-d88e-4e12-8fdc-ee7395f4c179\") " pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:07.863650 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.863546 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgsr6\" (UniqueName: \"kubernetes.io/projected/477b13c6-d88e-4e12-8fdc-ee7395f4c179-kube-api-access-pgsr6\") pod \"maas-api-9dcfd9c48-lgrqf\" (UID: \"477b13c6-d88e-4e12-8fdc-ee7395f4c179\") " pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:07.863650 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:12:07.863555 2571 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 02 19:12:07.863650 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:12:07.863630 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/477b13c6-d88e-4e12-8fdc-ee7395f4c179-maas-api-tls podName:477b13c6-d88e-4e12-8fdc-ee7395f4c179 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:08.363610078 +0000 UTC m=+565.654946956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/477b13c6-d88e-4e12-8fdc-ee7395f4c179-maas-api-tls") pod "maas-api-9dcfd9c48-lgrqf" (UID: "477b13c6-d88e-4e12-8fdc-ee7395f4c179") : secret "maas-api-serving-cert" not found Jul 02 19:12:07.864168 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.864148 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/477b13c6-d88e-4e12-8fdc-ee7395f4c179-openshift-service-ca\") pod \"maas-api-9dcfd9c48-lgrqf\" (UID: \"477b13c6-d88e-4e12-8fdc-ee7395f4c179\") " pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:07.872800 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:07.872774 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgsr6\" (UniqueName: \"kubernetes.io/projected/477b13c6-d88e-4e12-8fdc-ee7395f4c179-kube-api-access-pgsr6\") pod \"maas-api-9dcfd9c48-lgrqf\" (UID: \"477b13c6-d88e-4e12-8fdc-ee7395f4c179\") " pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:08.245347 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.245309 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-gzg82"] Jul 02 19:12:08.249252 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.249226 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-gzg82" Jul 02 19:12:08.252384 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.252355 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pt4dx\"" Jul 02 19:12:08.252636 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.252599 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 02 19:12:08.261676 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.261648 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-gzg82"] Jul 02 19:12:08.294520 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.294486 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-nzb5l"] Jul 02 19:12:08.298019 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.297991 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" Jul 02 19:12:08.309440 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.309415 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-nzb5l"] Jul 02 19:12:08.368706 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.368669 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2955ab2a-8039-463f-8d59-52950e3afadd-plugins-config-volume\") pod \"payload-pre-processing-7964457696-nzb5l\" (UID: \"2955ab2a-8039-463f-8d59-52950e3afadd\") " pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" Jul 02 19:12:08.368900 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.368714 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx9ff\" (UniqueName: \"kubernetes.io/projected/a6ec0e33-fb2e-4756-83e7-8fec1176a82f-kube-api-access-cx9ff\") pod \"payload-processing-58f86c7798-gzg82\" (UID: \"a6ec0e33-fb2e-4756-83e7-8fec1176a82f\") " pod="openshift-ingress/payload-processing-58f86c7798-gzg82" Jul 02 19:12:08.368900 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.368740 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv2zj\" (UniqueName: \"kubernetes.io/projected/2955ab2a-8039-463f-8d59-52950e3afadd-kube-api-access-wv2zj\") pod \"payload-pre-processing-7964457696-nzb5l\" (UID: \"2955ab2a-8039-463f-8d59-52950e3afadd\") " pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" Jul 02 19:12:08.368900 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.368854 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/477b13c6-d88e-4e12-8fdc-ee7395f4c179-maas-api-tls\") pod \"maas-api-9dcfd9c48-lgrqf\" (UID: \"477b13c6-d88e-4e12-8fdc-ee7395f4c179\") " pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:08.369030 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.368938 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ec0e33-fb2e-4756-83e7-8fec1176a82f-plugins-config-volume\") pod \"payload-processing-58f86c7798-gzg82\" (UID: \"a6ec0e33-fb2e-4756-83e7-8fec1176a82f\") " pod="openshift-ingress/payload-processing-58f86c7798-gzg82" Jul 02 19:12:08.371362 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.371331 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/477b13c6-d88e-4e12-8fdc-ee7395f4c179-maas-api-tls\") pod \"maas-api-9dcfd9c48-lgrqf\" (UID: \"477b13c6-d88e-4e12-8fdc-ee7395f4c179\") " pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:08.470206 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.470169 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ec0e33-fb2e-4756-83e7-8fec1176a82f-plugins-config-volume\") pod \"payload-processing-58f86c7798-gzg82\" (UID: \"a6ec0e33-fb2e-4756-83e7-8fec1176a82f\") " pod="openshift-ingress/payload-processing-58f86c7798-gzg82" Jul 02 19:12:08.470418 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.470250 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2955ab2a-8039-463f-8d59-52950e3afadd-plugins-config-volume\") pod \"payload-pre-processing-7964457696-nzb5l\" (UID: \"2955ab2a-8039-463f-8d59-52950e3afadd\") " pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" Jul 02 19:12:08.470418 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.470269 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9ff\" (UniqueName: \"kubernetes.io/projected/a6ec0e33-fb2e-4756-83e7-8fec1176a82f-kube-api-access-cx9ff\") pod \"payload-processing-58f86c7798-gzg82\" (UID: \"a6ec0e33-fb2e-4756-83e7-8fec1176a82f\") " pod="openshift-ingress/payload-processing-58f86c7798-gzg82" Jul 02 19:12:08.470418 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.470285 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv2zj\" (UniqueName: \"kubernetes.io/projected/2955ab2a-8039-463f-8d59-52950e3afadd-kube-api-access-wv2zj\") pod \"payload-pre-processing-7964457696-nzb5l\" (UID: \"2955ab2a-8039-463f-8d59-52950e3afadd\") " pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" Jul 02 19:12:08.470895 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.470872 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ec0e33-fb2e-4756-83e7-8fec1176a82f-plugins-config-volume\") pod \"payload-processing-58f86c7798-gzg82\" (UID: \"a6ec0e33-fb2e-4756-83e7-8fec1176a82f\") " pod="openshift-ingress/payload-processing-58f86c7798-gzg82" Jul 02 19:12:08.470994 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.470976 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2955ab2a-8039-463f-8d59-52950e3afadd-plugins-config-volume\") pod \"payload-pre-processing-7964457696-nzb5l\" (UID: \"2955ab2a-8039-463f-8d59-52950e3afadd\") " pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" Jul 02 19:12:08.484339 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.484308 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv2zj\" (UniqueName: \"kubernetes.io/projected/2955ab2a-8039-463f-8d59-52950e3afadd-kube-api-access-wv2zj\") pod \"payload-pre-processing-7964457696-nzb5l\" (UID: \"2955ab2a-8039-463f-8d59-52950e3afadd\") " pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" Jul 02 19:12:08.485510 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.485492 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx9ff\" (UniqueName: \"kubernetes.io/projected/a6ec0e33-fb2e-4756-83e7-8fec1176a82f-kube-api-access-cx9ff\") pod \"payload-processing-58f86c7798-gzg82\" (UID: \"a6ec0e33-fb2e-4756-83e7-8fec1176a82f\") " pod="openshift-ingress/payload-processing-58f86c7798-gzg82" Jul 02 19:12:08.562745 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.562705 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-gzg82" Jul 02 19:12:08.594135 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.594094 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:08.610587 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.610555 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" Jul 02 19:12:08.738858 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.738690 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-gzg82"] Jul 02 19:12:08.741056 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:12:08.741024 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ec0e33_fb2e_4756_83e7_8fec1176a82f.slice/crio-2910db55ea01c7ca924c02a4151a4c396e7bc6ef9c4870b03b728ba0333ce1e0 WatchSource:0}: Error finding container 2910db55ea01c7ca924c02a4151a4c396e7bc6ef9c4870b03b728ba0333ce1e0: Status 404 returned error can't find the container with id 2910db55ea01c7ca924c02a4151a4c396e7bc6ef9c4870b03b728ba0333ce1e0 Jul 02 19:12:08.778006 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.777975 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-9dcfd9c48-lgrqf"] Jul 02 19:12:08.779261 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:12:08.779234 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477b13c6_d88e_4e12_8fdc_ee7395f4c179.slice/crio-8c8c7b28c1b2a961ff201223e5afc412ab7e309d46239653bf199243047f4441 WatchSource:0}: Error finding container 8c8c7b28c1b2a961ff201223e5afc412ab7e309d46239653bf199243047f4441: Status 404 returned error can't find the container with id 8c8c7b28c1b2a961ff201223e5afc412ab7e309d46239653bf199243047f4441 Jul 02 19:12:08.808769 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:08.808744 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-nzb5l"] Jul 02 19:12:08.810348 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:12:08.810321 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2955ab2a_8039_463f_8d59_52950e3afadd.slice/crio-e15873309d17353befcd4c9219c294572d6ef2838caf4d36f54cc8caee93ab96 WatchSource:0}: Error finding container e15873309d17353befcd4c9219c294572d6ef2838caf4d36f54cc8caee93ab96: Status 404 returned error can't find the container with id e15873309d17353befcd4c9219c294572d6ef2838caf4d36f54cc8caee93ab96 Jul 02 19:12:09.054030 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:09.053988 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-9dcfd9c48-lgrqf" event={"ID":"477b13c6-d88e-4e12-8fdc-ee7395f4c179","Type":"ContainerStarted","Data":"8c8c7b28c1b2a961ff201223e5afc412ab7e309d46239653bf199243047f4441"} Jul 02 19:12:09.055557 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:09.055523 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-gzg82" event={"ID":"a6ec0e33-fb2e-4756-83e7-8fec1176a82f","Type":"ContainerStarted","Data":"2910db55ea01c7ca924c02a4151a4c396e7bc6ef9c4870b03b728ba0333ce1e0"} Jul 02 19:12:09.057409 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:09.057380 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" event={"ID":"2955ab2a-8039-463f-8d59-52950e3afadd","Type":"ContainerStarted","Data":"e15873309d17353befcd4c9219c294572d6ef2838caf4d36f54cc8caee93ab96"} Jul 02 19:12:12.080145 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:12.080103 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-9dcfd9c48-lgrqf" event={"ID":"477b13c6-d88e-4e12-8fdc-ee7395f4c179","Type":"ContainerStarted","Data":"4b8f2e1b470ee3ebe43fdf55705a2fe045f53b29ff3a9bcd9f0e7e0ae1edb403"} Jul 02 19:12:12.080642 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:12.080240 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:12.081642 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:12.081623 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-gzg82" event={"ID":"a6ec0e33-fb2e-4756-83e7-8fec1176a82f","Type":"ContainerStarted","Data":"a537bf7de14404083151a4ef777a1eaace911873da2b96708e7ea4be8eb51f0e"} Jul 02 19:12:12.081745 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:12.081709 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-gzg82" Jul 02 19:12:12.083054 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:12.083036 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" event={"ID":"2955ab2a-8039-463f-8d59-52950e3afadd","Type":"ContainerStarted","Data":"9bf1938ed8965563c53a796413fee36650cad4e15bfeaef3da7cb9ba2fc3a6c3"} Jul 02 19:12:12.083178 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:12.083091 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" Jul 02 19:12:12.099448 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:12.099392 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-9dcfd9c48-lgrqf" podStartSLOduration=2.898540037 podStartE2EDuration="5.099377019s" totalImagesPullingTime="2.200836982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:12.097615441 +0000 UTC m=+569.388952390" watchObservedRunningTime="2026-07-02 19:12:12.099377019 +0000 UTC m=+569.390713918" Jul 02 19:12:12.113589 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:12.113532 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-gzg82" podStartSLOduration=1.871547021 podStartE2EDuration="4.113512406s" totalImagesPullingTime="2.241965385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:12.113279133 +0000 UTC m=+569.404616034" watchObservedRunningTime="2026-07-02 19:12:12.113512406 +0000 UTC m=+569.404849304" Jul 02 19:12:12.131690 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:12.131621 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" podStartSLOduration=1.961607474 podStartE2EDuration="4.131606755s" totalImagesPullingTime="2.169999281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:12.130535477 +0000 UTC m=+569.421872377" watchObservedRunningTime="2026-07-02 19:12:12.131606755 +0000 UTC m=+569.422943656" Jul 02 19:12:16.106477 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:16.106441 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2515057-7616-44fa-8b3c-5878008819af" containerID="bc930632f6b33dc9be969877df39841466fdb50e12c78c6e38bb25b8cedbe361" exitCode=0 Jul 02 19:12:16.106939 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:16.106512 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" event={"ID":"d2515057-7616-44fa-8b3c-5878008819af","Type":"ContainerDied","Data":"bc930632f6b33dc9be969877df39841466fdb50e12c78c6e38bb25b8cedbe361"} Jul 02 19:12:16.106939 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:16.106904 2571 scope.go:117] "RemoveContainer" containerID="bc930632f6b33dc9be969877df39841466fdb50e12c78c6e38bb25b8cedbe361" Jul 02 19:12:16.107960 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:16.107927 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:12:16.108022 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:16.108008 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:12:16.108058 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:16.108042 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 19:12:17.114308 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:17.114276 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" event={"ID":"d2515057-7616-44fa-8b3c-5878008819af","Type":"ContainerStarted","Data":"b48add4aed13a217ac18e771546130d1a509813dc4aae4872830f20bdb22e733"} Jul 02 19:12:17.133127 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:17.133057 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:12:17.133621 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:17.133591 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-bqr69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 02 19:12:17.133713 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:17.133649 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" podUID="d2515057-7616-44fa-8b3c-5878008819af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 02 19:12:18.094225 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:18.094195 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-9dcfd9c48-lgrqf" Jul 02 19:12:18.133260 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:18.133207 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-bqr69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 02 19:12:18.133750 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:18.133295 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" podUID="d2515057-7616-44fa-8b3c-5878008819af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 02 19:12:19.133293 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:19.133243 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-bqr69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 02 19:12:19.133800 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:19.133321 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" podUID="d2515057-7616-44fa-8b3c-5878008819af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 02 19:12:20.133408 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:20.133374 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-bqr69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 02 19:12:20.134427 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:20.133995 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" podUID="d2515057-7616-44fa-8b3c-5878008819af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 02 19:12:21.145866 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:21.145835 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:12:21.146309 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:21.146113 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:12:21.146878 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:21.146862 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bqr69" Jul 02 19:12:23.090168 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:23.090138 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-gzg82" Jul 02 19:12:23.090612 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:23.090505 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-nzb5l" Jul 02 19:12:26.788493 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.788461 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m"] Jul 02 19:12:26.793165 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.793145 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.796544 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.796523 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 19:12:26.796674 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.796524 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 19:12:26.797285 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.797272 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mwzqz\"" Jul 02 19:12:26.797418 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.797399 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 02 19:12:26.802111 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.802061 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m"] Jul 02 19:12:26.840456 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.840413 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.840456 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.840452 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.840716 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.840485 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.840716 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.840512 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stxrz\" (UniqueName: \"kubernetes.io/projected/247553b6-f84b-4f02-a7ef-5b1f99909113-kube-api-access-stxrz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.840716 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.840586 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.840716 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.840690 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/247553b6-f84b-4f02-a7ef-5b1f99909113-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.941615 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.941573 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.941615 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.941608 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.941859 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.941635 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.941859 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.941661 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stxrz\" (UniqueName: \"kubernetes.io/projected/247553b6-f84b-4f02-a7ef-5b1f99909113-kube-api-access-stxrz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.941859 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.941718 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.941859 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.941823 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/247553b6-f84b-4f02-a7ef-5b1f99909113-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.942057 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.942029 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.942137 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.942112 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.942257 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.942229 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.944016 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.943996 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/247553b6-f84b-4f02-a7ef-5b1f99909113-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.944200 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.944183 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/247553b6-f84b-4f02-a7ef-5b1f99909113-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:26.951323 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:26.951297 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stxrz\" (UniqueName: \"kubernetes.io/projected/247553b6-f84b-4f02-a7ef-5b1f99909113-kube-api-access-stxrz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m\" (UID: \"247553b6-f84b-4f02-a7ef-5b1f99909113\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:27.105798 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:27.105695 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:27.261319 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:27.261295 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m"] Jul 02 19:12:27.263569 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:12:27.263541 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod247553b6_f84b_4f02_a7ef_5b1f99909113.slice/crio-ef3afc1bbb60fc7a7f4ec179c80ac97245c1cba54e1a8303c3cecd1cb65c6324 WatchSource:0}: Error finding container ef3afc1bbb60fc7a7f4ec179c80ac97245c1cba54e1a8303c3cecd1cb65c6324: Status 404 returned error can't find the container with id ef3afc1bbb60fc7a7f4ec179c80ac97245c1cba54e1a8303c3cecd1cb65c6324 Jul 02 19:12:28.186359 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:28.186319 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" event={"ID":"247553b6-f84b-4f02-a7ef-5b1f99909113","Type":"ContainerStarted","Data":"ef3afc1bbb60fc7a7f4ec179c80ac97245c1cba54e1a8303c3cecd1cb65c6324"} Jul 02 19:12:29.193826 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:29.193783 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" event={"ID":"247553b6-f84b-4f02-a7ef-5b1f99909113","Type":"ContainerStarted","Data":"51146b55a9dec0a90d71a2b3ce1a17986c3e0de14f80d814ffc59058f729ced7"} Jul 02 19:12:29.213088 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:29.213025 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" podStartSLOduration=2.03803564 podStartE2EDuration="3.213009901s" totalImagesPullingTime="1.174974261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:29.211518415 +0000 UTC m=+586.502855312" watchObservedRunningTime="2026-07-02 19:12:29.213009901 +0000 UTC m=+586.504346801" Jul 02 19:12:34.606165 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.606130 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc"] Jul 02 19:12:34.610199 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.610181 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.613011 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.612993 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 02 19:12:34.623273 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.623247 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc"] Jul 02 19:12:34.721610 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.721571 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.721610 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.721607 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz6zm\" (UniqueName: \"kubernetes.io/projected/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-kube-api-access-xz6zm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.721825 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.721639 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.721825 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.721658 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.721825 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.721754 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.721825 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.721794 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.823063 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.823026 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.823063 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.823063 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz6zm\" (UniqueName: \"kubernetes.io/projected/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-kube-api-access-xz6zm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.823348 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.823099 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.823348 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.823119 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.823348 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.823159 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.823348 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.823177 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.823641 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.823622 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.823696 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.823662 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.823752 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.823731 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.825599 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.825571 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.825725 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.825710 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.833326 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.833296 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz6zm\" (UniqueName: \"kubernetes.io/projected/ea4bc6a9-7565-4a99-b702-fdba2141ba9e-kube-api-access-xz6zm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lx9zc\" (UID: \"ea4bc6a9-7565-4a99-b702-fdba2141ba9e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:34.921027 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:34.920928 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:35.069059 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:35.069024 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc"] Jul 02 19:12:35.070609 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:12:35.070580 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4bc6a9_7565_4a99_b702_fdba2141ba9e.slice/crio-1802ba00d2e72daddae1b2903b9d643aa968802112276970bab0d01ed1947a20 WatchSource:0}: Error finding container 1802ba00d2e72daddae1b2903b9d643aa968802112276970bab0d01ed1947a20: Status 404 returned error can't find the container with id 1802ba00d2e72daddae1b2903b9d643aa968802112276970bab0d01ed1947a20 Jul 02 19:12:35.228141 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:35.228045 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" event={"ID":"ea4bc6a9-7565-4a99-b702-fdba2141ba9e","Type":"ContainerStarted","Data":"1802ba00d2e72daddae1b2903b9d643aa968802112276970bab0d01ed1947a20"} Jul 02 19:12:36.237336 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:36.237295 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" event={"ID":"ea4bc6a9-7565-4a99-b702-fdba2141ba9e","Type":"ContainerStarted","Data":"edec82913e2537ab0c54afabc63c38bed3fcdcae7b66c5648554e07f124a2a9d"} Jul 02 19:12:36.277268 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:36.277215 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" podStartSLOduration=2.096051932 podStartE2EDuration="2.277196504s" totalImagesPullingTime="181.144572ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:36.275095682 +0000 UTC m=+593.566432574" watchObservedRunningTime="2026-07-02 19:12:36.277196504 +0000 UTC m=+593.568533403" Jul 02 19:12:37.106878 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:37.106841 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:37.107066 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:37.106906 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:37.119112 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:37.119086 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:37.254238 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:37.254209 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m" Jul 02 19:12:44.921212 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:44.921169 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:44.921629 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:44.921243 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:44.934040 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:44.934012 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:12:45.299346 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:12:45.299320 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lx9zc" Jul 02 19:13:11.872029 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:11.871952 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb"] Jul 02 19:13:11.875260 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:11.875237 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:11.877857 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:11.877836 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 02 19:13:11.886295 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:11.886271 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb"] Jul 02 19:13:11.976782 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:11.976740 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:11.976782 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:11.976788 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:11.977002 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:11.976805 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsz9m\" (UniqueName: \"kubernetes.io/projected/e0d2b639-8daf-4887-b3aa-f3529242831f-kube-api-access-wsz9m\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:11.977002 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:11.976897 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:11.977002 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:11.976936 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0d2b639-8daf-4887-b3aa-f3529242831f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:11.977002 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:11.976966 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.078193 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.078154 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.078379 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.078201 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.078379 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.078226 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsz9m\" (UniqueName: \"kubernetes.io/projected/e0d2b639-8daf-4887-b3aa-f3529242831f-kube-api-access-wsz9m\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.078379 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.078274 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.078379 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.078309 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0d2b639-8daf-4887-b3aa-f3529242831f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.078379 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.078348 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.078682 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.078660 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.078745 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.078685 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.078745 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.078721 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.080452 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.080422 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0d2b639-8daf-4887-b3aa-f3529242831f-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.080689 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.080671 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0d2b639-8daf-4887-b3aa-f3529242831f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.086771 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.086745 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsz9m\" (UniqueName: \"kubernetes.io/projected/e0d2b639-8daf-4887-b3aa-f3529242831f-kube-api-access-wsz9m\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb\" (UID: \"e0d2b639-8daf-4887-b3aa-f3529242831f\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.185967 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.185872 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:12.323950 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.323928 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb"] Jul 02 19:13:12.325676 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:13:12.325647 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0d2b639_8daf_4887_b3aa_f3529242831f.slice/crio-296c5bc91b3fee091ab2e7a2e85037b4cc8a21d504801aa87e28c5c9fa7d3acc WatchSource:0}: Error finding container 296c5bc91b3fee091ab2e7a2e85037b4cc8a21d504801aa87e28c5c9fa7d3acc: Status 404 returned error can't find the container with id 296c5bc91b3fee091ab2e7a2e85037b4cc8a21d504801aa87e28c5c9fa7d3acc Jul 02 19:13:12.327116 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.327097 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:13:12.451413 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:12.451331 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" event={"ID":"e0d2b639-8daf-4887-b3aa-f3529242831f","Type":"ContainerStarted","Data":"296c5bc91b3fee091ab2e7a2e85037b4cc8a21d504801aa87e28c5c9fa7d3acc"} Jul 02 19:13:13.458788 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:13.458750 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" event={"ID":"e0d2b639-8daf-4887-b3aa-f3529242831f","Type":"ContainerStarted","Data":"ac98b679cdf4eefaa4f4f2f49f2c46941f2af643333d5597d85ced6edfac382e"} Jul 02 19:13:13.485423 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:13.485365 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" podStartSLOduration=2.312849237 podStartE2EDuration="2.485347292s" totalImagesPullingTime="172.498055ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:13.482512828 +0000 UTC m=+630.773849718" watchObservedRunningTime="2026-07-02 19:13:13.485347292 +0000 UTC m=+630.776684192" Jul 02 19:13:22.186763 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:22.186726 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:22.187272 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:22.186775 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:22.199249 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:22.199217 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:13:22.520141 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:13:22.520117 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb" Jul 02 19:14:33.462214 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.462178 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-845f95f44b-dmj2w"] Jul 02 19:14:33.465750 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.465733 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-845f95f44b-dmj2w" Jul 02 19:14:33.473718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.473692 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-845f95f44b-dmj2w"] Jul 02 19:14:33.531476 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.531440 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1af8b5a9-e426-4a3c-9697-df2256ab0e5c-tls-cert\") pod \"authorino-845f95f44b-dmj2w\" (UID: \"1af8b5a9-e426-4a3c-9697-df2256ab0e5c\") " pod="kuadrant-system/authorino-845f95f44b-dmj2w" Jul 02 19:14:33.531476 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.531482 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwlnx\" (UniqueName: \"kubernetes.io/projected/1af8b5a9-e426-4a3c-9697-df2256ab0e5c-kube-api-access-gwlnx\") pod \"authorino-845f95f44b-dmj2w\" (UID: \"1af8b5a9-e426-4a3c-9697-df2256ab0e5c\") " pod="kuadrant-system/authorino-845f95f44b-dmj2w" Jul 02 19:14:33.632586 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.632546 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1af8b5a9-e426-4a3c-9697-df2256ab0e5c-tls-cert\") pod \"authorino-845f95f44b-dmj2w\" (UID: \"1af8b5a9-e426-4a3c-9697-df2256ab0e5c\") " pod="kuadrant-system/authorino-845f95f44b-dmj2w" Jul 02 19:14:33.632586 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.632584 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwlnx\" (UniqueName: \"kubernetes.io/projected/1af8b5a9-e426-4a3c-9697-df2256ab0e5c-kube-api-access-gwlnx\") pod \"authorino-845f95f44b-dmj2w\" (UID: \"1af8b5a9-e426-4a3c-9697-df2256ab0e5c\") " pod="kuadrant-system/authorino-845f95f44b-dmj2w" Jul 02 19:14:33.635048 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.635021 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1af8b5a9-e426-4a3c-9697-df2256ab0e5c-tls-cert\") pod \"authorino-845f95f44b-dmj2w\" (UID: \"1af8b5a9-e426-4a3c-9697-df2256ab0e5c\") " pod="kuadrant-system/authorino-845f95f44b-dmj2w" Jul 02 19:14:33.641204 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.641180 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwlnx\" (UniqueName: \"kubernetes.io/projected/1af8b5a9-e426-4a3c-9697-df2256ab0e5c-kube-api-access-gwlnx\") pod \"authorino-845f95f44b-dmj2w\" (UID: \"1af8b5a9-e426-4a3c-9697-df2256ab0e5c\") " pod="kuadrant-system/authorino-845f95f44b-dmj2w" Jul 02 19:14:33.776605 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.776572 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-845f95f44b-dmj2w" Jul 02 19:14:33.913485 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.913455 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-845f95f44b-dmj2w"] Jul 02 19:14:33.914677 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:14:33.914652 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1af8b5a9_e426_4a3c_9697_df2256ab0e5c.slice/crio-e7658be76f9129028a33b49c16af3a4d196c083f3ffde6cbfdc4d7a148fcd708 WatchSource:0}: Error finding container e7658be76f9129028a33b49c16af3a4d196c083f3ffde6cbfdc4d7a148fcd708: Status 404 returned error can't find the container with id e7658be76f9129028a33b49c16af3a4d196c083f3ffde6cbfdc4d7a148fcd708 Jul 02 19:14:33.922295 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:33.922266 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-845f95f44b-dmj2w" event={"ID":"1af8b5a9-e426-4a3c-9697-df2256ab0e5c","Type":"ContainerStarted","Data":"e7658be76f9129028a33b49c16af3a4d196c083f3ffde6cbfdc4d7a148fcd708"} Jul 02 19:14:34.932764 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:34.932655 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-845f95f44b-dmj2w" event={"ID":"1af8b5a9-e426-4a3c-9697-df2256ab0e5c","Type":"ContainerStarted","Data":"89712d3171358cc786ecf99571f84bc46534fb6c9631dbe1d639adc7a90dd820"} Jul 02 19:14:34.964936 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:34.964880 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-845f95f44b-dmj2w" podStartSLOduration=1.572328041 podStartE2EDuration="1.964856917s" totalImagesPullingTime="392.528876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:34.959091015 +0000 UTC m=+712.250427912" watchObservedRunningTime="2026-07-02 19:14:34.964856917 +0000 UTC m=+712.256193856" Jul 02 19:14:35.056057 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.056012 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-89b864fc5-78k8q"] Jul 02 19:14:35.056337 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.056288 2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-89b864fc5-78k8q" podUID="6d39c7a9-1f3b-43a2-ae78-d55a3c176be6" containerName="authorino" containerID="cri-o://6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18" gracePeriod=30 Jul 02 19:14:35.330679 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.330654 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-89b864fc5-78k8q" Jul 02 19:14:35.352306 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.352270 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq4ls\" (UniqueName: \"kubernetes.io/projected/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-kube-api-access-rq4ls\") pod \"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6\" (UID: \"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6\") " Jul 02 19:14:35.352306 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.352326 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-tls-cert\") pod \"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6\" (UID: \"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6\") " Jul 02 19:14:35.354655 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.354624 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-kube-api-access-rq4ls" (OuterVolumeSpecName: "kube-api-access-rq4ls") pod "6d39c7a9-1f3b-43a2-ae78-d55a3c176be6" (UID: "6d39c7a9-1f3b-43a2-ae78-d55a3c176be6"). InnerVolumeSpecName "kube-api-access-rq4ls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:14:35.366223 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.366192 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6d39c7a9-1f3b-43a2-ae78-d55a3c176be6" (UID: "6d39c7a9-1f3b-43a2-ae78-d55a3c176be6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:14:35.454959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.453703 2571 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-tls-cert\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:14:35.454959 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.453740 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq4ls\" (UniqueName: \"kubernetes.io/projected/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6-kube-api-access-rq4ls\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:14:35.942518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.942421 2571 generic.go:356] "Generic (PLEG): container finished" podID="6d39c7a9-1f3b-43a2-ae78-d55a3c176be6" containerID="6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18" exitCode=0 Jul 02 19:14:35.942518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.942460 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-89b864fc5-78k8q" Jul 02 19:14:35.942518 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.942498 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-89b864fc5-78k8q" event={"ID":"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6","Type":"ContainerDied","Data":"6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18"} Jul 02 19:14:35.943124 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.942540 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-89b864fc5-78k8q" event={"ID":"6d39c7a9-1f3b-43a2-ae78-d55a3c176be6","Type":"ContainerDied","Data":"36433246d629b92eca55f04036424b0f168c4907da346767d980853286d4bce7"} Jul 02 19:14:35.943124 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.942562 2571 scope.go:117] "RemoveContainer" containerID="6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18" Jul 02 19:14:35.954640 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.954619 2571 scope.go:117] "RemoveContainer" containerID="6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18" Jul 02 19:14:35.954976 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:14:35.954953 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18\": container with ID starting with 6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18 not found: ID does not exist" containerID="6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18" Jul 02 19:14:35.955059 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.954983 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18"} err="failed to get container status \"6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18\": rpc error: code = NotFound desc = could not find container \"6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18\": container with ID starting with 6b7a92de2efe4da3fb1d3efc419531871b921bec04e0f41b49e5c98f103dbf18 not found: ID does not exist" Jul 02 19:14:35.965032 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.965005 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-89b864fc5-78k8q"] Jul 02 19:14:35.973035 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:35.973006 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-89b864fc5-78k8q"] Jul 02 19:14:37.238301 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:14:37.238267 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d39c7a9-1f3b-43a2-ae78-d55a3c176be6" path="/var/lib/kubelet/pods/6d39c7a9-1f3b-43a2-ae78-d55a3c176be6/volumes" Jul 02 19:15:48.215559 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:48.215522 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85fc68f447-zkm6j"] Jul 02 19:15:48.216122 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:48.215724 2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-85fc68f447-zkm6j" podUID="7b1875bc-fb16-4dd4-8d1f-c9848049c3c7" containerName="manager" containerID="cri-o://94b6348aca24b6ee0f084456630b1afcc301a615d1862451a2798da3be0fdacc" gracePeriod=30 Jul 02 19:15:48.365016 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:48.364984 2571 generic.go:356] "Generic (PLEG): container finished" podID="7b1875bc-fb16-4dd4-8d1f-c9848049c3c7" containerID="94b6348aca24b6ee0f084456630b1afcc301a615d1862451a2798da3be0fdacc" exitCode=0 Jul 02 19:15:48.365198 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:48.365050 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85fc68f447-zkm6j" event={"ID":"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7","Type":"ContainerDied","Data":"94b6348aca24b6ee0f084456630b1afcc301a615d1862451a2798da3be0fdacc"} Jul 02 19:15:48.472123 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:48.472051 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:15:48.559061 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:48.559033 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2q6dc\" (UniqueName: \"kubernetes.io/projected/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-kube-api-access-2q6dc\") pod \"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7\" (UID: \"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7\") " Jul 02 19:15:48.559239 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:48.559132 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-webhook-cert\") pod \"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7\" (UID: \"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7\") " Jul 02 19:15:48.561139 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:48.561107 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7b1875bc-fb16-4dd4-8d1f-c9848049c3c7" (UID: "7b1875bc-fb16-4dd4-8d1f-c9848049c3c7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:15:48.561139 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:48.561108 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-kube-api-access-2q6dc" (OuterVolumeSpecName: "kube-api-access-2q6dc") pod "7b1875bc-fb16-4dd4-8d1f-c9848049c3c7" (UID: "7b1875bc-fb16-4dd4-8d1f-c9848049c3c7"). InnerVolumeSpecName "kube-api-access-2q6dc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:15:48.660479 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:48.660444 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q6dc\" (UniqueName: \"kubernetes.io/projected/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-kube-api-access-2q6dc\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:15:48.660479 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:48.660471 2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7-webhook-cert\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:15:49.371728 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:49.371698 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85fc68f447-zkm6j" Jul 02 19:15:49.372312 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:49.371702 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85fc68f447-zkm6j" event={"ID":"7b1875bc-fb16-4dd4-8d1f-c9848049c3c7","Type":"ContainerDied","Data":"2013eaf041d72bd7df2a7b8b2f55a2d6e70cf3d9c6a50897161f1e793fc24d47"} Jul 02 19:15:49.372312 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:49.371807 2571 scope.go:117] "RemoveContainer" containerID="94b6348aca24b6ee0f084456630b1afcc301a615d1862451a2798da3be0fdacc" Jul 02 19:15:49.396309 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:49.396276 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85fc68f447-zkm6j"] Jul 02 19:15:49.404822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:49.404795 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-85fc68f447-zkm6j"] Jul 02 19:15:50.184163 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.184123 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85fc68f447-vjvhf"] Jul 02 19:15:50.184583 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.184571 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d39c7a9-1f3b-43a2-ae78-d55a3c176be6" containerName="authorino" Jul 02 19:15:50.184628 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.184584 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d39c7a9-1f3b-43a2-ae78-d55a3c176be6" containerName="authorino" Jul 02 19:15:50.184628 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.184595 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b1875bc-fb16-4dd4-8d1f-c9848049c3c7" containerName="manager" Jul 02 19:15:50.184628 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.184601 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b1875bc-fb16-4dd4-8d1f-c9848049c3c7" containerName="manager" Jul 02 19:15:50.184726 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.184660 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d39c7a9-1f3b-43a2-ae78-d55a3c176be6" containerName="authorino" Jul 02 19:15:50.184726 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.184668 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b1875bc-fb16-4dd4-8d1f-c9848049c3c7" containerName="manager" Jul 02 19:15:50.189278 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.189259 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85fc68f447-vjvhf" Jul 02 19:15:50.193045 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.193015 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 02 19:15:50.193203 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.193087 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 02 19:15:50.193203 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.193102 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6xt7d\"" Jul 02 19:15:50.196162 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.196139 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85fc68f447-vjvhf"] Jul 02 19:15:50.275817 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.275774 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ba6d0e2-f568-4e1e-abb9-415c27a00439-webhook-cert\") pod \"maas-controller-85fc68f447-vjvhf\" (UID: \"6ba6d0e2-f568-4e1e-abb9-415c27a00439\") " pod="opendatahub/maas-controller-85fc68f447-vjvhf" Jul 02 19:15:50.275999 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.275825 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bn4r\" (UniqueName: \"kubernetes.io/projected/6ba6d0e2-f568-4e1e-abb9-415c27a00439-kube-api-access-2bn4r\") pod \"maas-controller-85fc68f447-vjvhf\" (UID: \"6ba6d0e2-f568-4e1e-abb9-415c27a00439\") " pod="opendatahub/maas-controller-85fc68f447-vjvhf" Jul 02 19:15:50.377113 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.377050 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ba6d0e2-f568-4e1e-abb9-415c27a00439-webhook-cert\") pod \"maas-controller-85fc68f447-vjvhf\" (UID: \"6ba6d0e2-f568-4e1e-abb9-415c27a00439\") " pod="opendatahub/maas-controller-85fc68f447-vjvhf" Jul 02 19:15:50.377654 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.377625 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bn4r\" (UniqueName: \"kubernetes.io/projected/6ba6d0e2-f568-4e1e-abb9-415c27a00439-kube-api-access-2bn4r\") pod \"maas-controller-85fc68f447-vjvhf\" (UID: \"6ba6d0e2-f568-4e1e-abb9-415c27a00439\") " pod="opendatahub/maas-controller-85fc68f447-vjvhf" Jul 02 19:15:50.379451 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.379430 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ba6d0e2-f568-4e1e-abb9-415c27a00439-webhook-cert\") pod \"maas-controller-85fc68f447-vjvhf\" (UID: \"6ba6d0e2-f568-4e1e-abb9-415c27a00439\") " pod="opendatahub/maas-controller-85fc68f447-vjvhf" Jul 02 19:15:50.386146 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.386116 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bn4r\" (UniqueName: \"kubernetes.io/projected/6ba6d0e2-f568-4e1e-abb9-415c27a00439-kube-api-access-2bn4r\") pod \"maas-controller-85fc68f447-vjvhf\" (UID: \"6ba6d0e2-f568-4e1e-abb9-415c27a00439\") " pod="opendatahub/maas-controller-85fc68f447-vjvhf" Jul 02 19:15:50.501339 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.501247 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85fc68f447-vjvhf" Jul 02 19:15:50.640725 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:50.640702 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85fc68f447-vjvhf"] Jul 02 19:15:50.641979 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:15:50.641955 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba6d0e2_f568_4e1e_abb9_415c27a00439.slice/crio-813d9df3a179052fe0fef08ce9498c6e09787e6a2b37d8243312dda7a0034fc2 WatchSource:0}: Error finding container 813d9df3a179052fe0fef08ce9498c6e09787e6a2b37d8243312dda7a0034fc2: Status 404 returned error can't find the container with id 813d9df3a179052fe0fef08ce9498c6e09787e6a2b37d8243312dda7a0034fc2 Jul 02 19:15:51.238951 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:51.238920 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b1875bc-fb16-4dd4-8d1f-c9848049c3c7" path="/var/lib/kubelet/pods/7b1875bc-fb16-4dd4-8d1f-c9848049c3c7/volumes" Jul 02 19:15:51.388576 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:51.388467 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85fc68f447-vjvhf" event={"ID":"6ba6d0e2-f568-4e1e-abb9-415c27a00439","Type":"ContainerStarted","Data":"05551b44e549b485ced385016b83040a791c7ae5b898cbe92ceb10f5344d21db"} Jul 02 19:15:51.388576 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:51.388542 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85fc68f447-vjvhf" Jul 02 19:15:51.388576 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:51.388571 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85fc68f447-vjvhf" event={"ID":"6ba6d0e2-f568-4e1e-abb9-415c27a00439","Type":"ContainerStarted","Data":"813d9df3a179052fe0fef08ce9498c6e09787e6a2b37d8243312dda7a0034fc2"} Jul 02 19:15:51.407586 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:15:51.407535 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85fc68f447-vjvhf" podStartSLOduration=0.95707198 podStartE2EDuration="1.407515959s" totalImagesPullingTime="450.443979ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:51.405970309 +0000 UTC m=+788.697307219" watchObservedRunningTime="2026-07-02 19:15:51.407515959 +0000 UTC m=+788.698852859" Jul 02 19:16:02.400767 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:16:02.400733 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85fc68f447-vjvhf" Jul 02 19:42:36.466116 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:36.466081 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-139976271669264"] Jul 02 19:42:36.469681 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:36.469663 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669264" Jul 02 19:42:36.476898 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:36.476665 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139976271669264"] Jul 02 19:42:36.604878 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:36.604824 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j76w5\" (UniqueName: \"kubernetes.io/projected/86105643-405e-41a6-8f87-9c402bf474c3-kube-api-access-j76w5\") pod \"test-curl-7464-139976271669264\" (UID: \"86105643-405e-41a6-8f87-9c402bf474c3\") " pod="opendatahub/test-curl-7464-139976271669264" Jul 02 19:42:36.705831 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:36.705799 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j76w5\" (UniqueName: \"kubernetes.io/projected/86105643-405e-41a6-8f87-9c402bf474c3-kube-api-access-j76w5\") pod \"test-curl-7464-139976271669264\" (UID: \"86105643-405e-41a6-8f87-9c402bf474c3\") " pod="opendatahub/test-curl-7464-139976271669264" Jul 02 19:42:36.713855 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:36.713823 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j76w5\" (UniqueName: \"kubernetes.io/projected/86105643-405e-41a6-8f87-9c402bf474c3-kube-api-access-j76w5\") pod \"test-curl-7464-139976271669264\" (UID: \"86105643-405e-41a6-8f87-9c402bf474c3\") " pod="opendatahub/test-curl-7464-139976271669264" Jul 02 19:42:36.779443 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:36.779415 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669264" Jul 02 19:42:36.919238 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:36.919203 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139976271669264"] Jul 02 19:42:36.920770 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:42:36.920739 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86105643_405e_41a6_8f87_9c402bf474c3.slice/crio-a364935e2342d98272d2f809ae5d889dff813ffdda722a7376c6b0f992a39a49 WatchSource:0}: Error finding container a364935e2342d98272d2f809ae5d889dff813ffdda722a7376c6b0f992a39a49: Status 404 returned error can't find the container with id a364935e2342d98272d2f809ae5d889dff813ffdda722a7376c6b0f992a39a49 Jul 02 19:42:36.922024 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:36.922009 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:42:37.715949 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:37.715905 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139976271669264" event={"ID":"86105643-405e-41a6-8f87-9c402bf474c3","Type":"ContainerStarted","Data":"da1f11ffaa9ea28e9d3bb2f2ff13268c48a14df7eea230348b5da350360e1e52"} Jul 02 19:42:37.716342 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:37.715967 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139976271669264" event={"ID":"86105643-405e-41a6-8f87-9c402bf474c3","Type":"ContainerStarted","Data":"a364935e2342d98272d2f809ae5d889dff813ffdda722a7376c6b0f992a39a49"} Jul 02 19:42:37.731875 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:37.731829 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7464-139976271669264" podStartSLOduration=1.052620947 podStartE2EDuration="1.731814181s" totalImagesPullingTime="679.193234ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:42:37.730548874 +0000 UTC m=+2395.021885781" watchObservedRunningTime="2026-07-02 19:42:37.731814181 +0000 UTC m=+2395.023151117" Jul 02 19:42:37.848973 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:37.848942 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-139976271669264_86105643-405e-41a6-8f87-9c402bf474c3/test-curl-7464-139976271669264/0.log" Jul 02 19:42:38.722891 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:38.722856 2571 generic.go:356] "Generic (PLEG): container finished" podID="86105643-405e-41a6-8f87-9c402bf474c3" containerID="da1f11ffaa9ea28e9d3bb2f2ff13268c48a14df7eea230348b5da350360e1e52" exitCode=0 Jul 02 19:42:38.723299 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:38.722927 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139976271669264" event={"ID":"86105643-405e-41a6-8f87-9c402bf474c3","Type":"ContainerDied","Data":"da1f11ffaa9ea28e9d3bb2f2ff13268c48a14df7eea230348b5da350360e1e52"} Jul 02 19:42:39.871792 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:39.871762 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669264" Jul 02 19:42:39.911865 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:39.911825 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-139976271669264"] Jul 02 19:42:39.920771 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:39.917525 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-139976271669264"] Jul 02 19:42:39.937188 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:39.937158 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j76w5\" (UniqueName: \"kubernetes.io/projected/86105643-405e-41a6-8f87-9c402bf474c3-kube-api-access-j76w5\") pod \"86105643-405e-41a6-8f87-9c402bf474c3\" (UID: \"86105643-405e-41a6-8f87-9c402bf474c3\") " Jul 02 19:42:39.939199 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:39.939173 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86105643-405e-41a6-8f87-9c402bf474c3-kube-api-access-j76w5" (OuterVolumeSpecName: "kube-api-access-j76w5") pod "86105643-405e-41a6-8f87-9c402bf474c3" (UID: "86105643-405e-41a6-8f87-9c402bf474c3"). InnerVolumeSpecName "kube-api-access-j76w5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:40.037570 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.037537 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-139976271669488"] Jul 02 19:42:40.037955 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.037928 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j76w5\" (UniqueName: \"kubernetes.io/projected/86105643-405e-41a6-8f87-9c402bf474c3-kube-api-access-j76w5\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:42:40.038020 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.038007 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86105643-405e-41a6-8f87-9c402bf474c3" containerName="test-curl-7464-139976271669264" Jul 02 19:42:40.038059 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.038022 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="86105643-405e-41a6-8f87-9c402bf474c3" containerName="test-curl-7464-139976271669264" Jul 02 19:42:40.038119 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.038106 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="86105643-405e-41a6-8f87-9c402bf474c3" containerName="test-curl-7464-139976271669264" Jul 02 19:42:40.041386 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.041370 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669488" Jul 02 19:42:40.045295 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.045271 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139976271669488"] Jul 02 19:42:40.138650 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.138604 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p5z2\" (UniqueName: \"kubernetes.io/projected/8755e909-4076-40e4-ba16-5d559c7b981d-kube-api-access-4p5z2\") pod \"test-curl-7464-139976271669488\" (UID: \"8755e909-4076-40e4-ba16-5d559c7b981d\") " pod="opendatahub/test-curl-7464-139976271669488" Jul 02 19:42:40.239940 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.239904 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p5z2\" (UniqueName: \"kubernetes.io/projected/8755e909-4076-40e4-ba16-5d559c7b981d-kube-api-access-4p5z2\") pod \"test-curl-7464-139976271669488\" (UID: \"8755e909-4076-40e4-ba16-5d559c7b981d\") " pod="opendatahub/test-curl-7464-139976271669488" Jul 02 19:42:40.248064 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.248039 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p5z2\" (UniqueName: \"kubernetes.io/projected/8755e909-4076-40e4-ba16-5d559c7b981d-kube-api-access-4p5z2\") pod \"test-curl-7464-139976271669488\" (UID: \"8755e909-4076-40e4-ba16-5d559c7b981d\") " pod="opendatahub/test-curl-7464-139976271669488" Jul 02 19:42:40.352246 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.352147 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669488" Jul 02 19:42:40.695336 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.695309 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139976271669488"] Jul 02 19:42:40.696787 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:42:40.696759 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8755e909_4076_40e4_ba16_5d559c7b981d.slice/crio-30cb3ceae54f1f9e1ecd7e78be1784d7c8245ad29b7c475fbaa956906b994278 WatchSource:0}: Error finding container 30cb3ceae54f1f9e1ecd7e78be1784d7c8245ad29b7c475fbaa956906b994278: Status 404 returned error can't find the container with id 30cb3ceae54f1f9e1ecd7e78be1784d7c8245ad29b7c475fbaa956906b994278 Jul 02 19:42:40.736388 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.736349 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139976271669488" event={"ID":"8755e909-4076-40e4-ba16-5d559c7b981d","Type":"ContainerStarted","Data":"30cb3ceae54f1f9e1ecd7e78be1784d7c8245ad29b7c475fbaa956906b994278"} Jul 02 19:42:40.737541 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.737521 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669264" Jul 02 19:42:40.737655 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:40.737563 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a364935e2342d98272d2f809ae5d889dff813ffdda722a7376c6b0f992a39a49" Jul 02 19:42:40.747918 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:42:40.747891 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7464-139976271669264\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="86105643-405e-41a6-8f87-9c402bf474c3" pod="opendatahub/test-curl-7464-139976271669264" Jul 02 19:42:41.238761 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:41.238728 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86105643-405e-41a6-8f87-9c402bf474c3" path="/var/lib/kubelet/pods/86105643-405e-41a6-8f87-9c402bf474c3/volumes" Jul 02 19:42:41.744750 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:41.744716 2571 generic.go:356] "Generic (PLEG): container finished" podID="8755e909-4076-40e4-ba16-5d559c7b981d" containerID="2499acf24182be4da74394251981fd513d7a5413acf27071cfbe30cc44eeae12" exitCode=0 Jul 02 19:42:41.744940 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:41.744812 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139976271669488" event={"ID":"8755e909-4076-40e4-ba16-5d559c7b981d","Type":"ContainerDied","Data":"2499acf24182be4da74394251981fd513d7a5413acf27071cfbe30cc44eeae12"} Jul 02 19:42:42.897943 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:42.897918 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669488" Jul 02 19:42:42.917467 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:42.917440 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-139976271669488_8755e909-4076-40e4-ba16-5d559c7b981d/test-curl-7464-139976271669488/0.log" Jul 02 19:42:42.942158 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:42.942126 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-139976271669488"] Jul 02 19:42:42.947987 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:42.947961 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-139976271669488"] Jul 02 19:42:42.967411 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:42.967377 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4p5z2\" (UniqueName: \"kubernetes.io/projected/8755e909-4076-40e4-ba16-5d559c7b981d-kube-api-access-4p5z2\") pod \"8755e909-4076-40e4-ba16-5d559c7b981d\" (UID: \"8755e909-4076-40e4-ba16-5d559c7b981d\") " Jul 02 19:42:42.969482 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:42.969455 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8755e909-4076-40e4-ba16-5d559c7b981d-kube-api-access-4p5z2" (OuterVolumeSpecName: "kube-api-access-4p5z2") pod "8755e909-4076-40e4-ba16-5d559c7b981d" (UID: "8755e909-4076-40e4-ba16-5d559c7b981d"). InnerVolumeSpecName "kube-api-access-4p5z2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:43.068648 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.068617 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p5z2\" (UniqueName: \"kubernetes.io/projected/8755e909-4076-40e4-ba16-5d559c7b981d-kube-api-access-4p5z2\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:42:43.070112 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.070092 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-139976271669600"] Jul 02 19:42:43.070527 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.070515 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8755e909-4076-40e4-ba16-5d559c7b981d" containerName="test-curl-7464-139976271669488" Jul 02 19:42:43.070572 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.070529 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8755e909-4076-40e4-ba16-5d559c7b981d" containerName="test-curl-7464-139976271669488" Jul 02 19:42:43.070604 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.070591 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8755e909-4076-40e4-ba16-5d559c7b981d" containerName="test-curl-7464-139976271669488" Jul 02 19:42:43.073557 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.073540 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669600" Jul 02 19:42:43.080016 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.079990 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139976271669600"] Jul 02 19:42:43.169269 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.169224 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5cq5\" (UniqueName: \"kubernetes.io/projected/e7850257-5351-4dc2-b64e-6e1f5ee927bc-kube-api-access-r5cq5\") pod \"test-curl-7464-139976271669600\" (UID: \"e7850257-5351-4dc2-b64e-6e1f5ee927bc\") " pod="opendatahub/test-curl-7464-139976271669600" Jul 02 19:42:43.238881 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.238840 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8755e909-4076-40e4-ba16-5d559c7b981d" path="/var/lib/kubelet/pods/8755e909-4076-40e4-ba16-5d559c7b981d/volumes" Jul 02 19:42:43.269798 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.269767 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5cq5\" (UniqueName: \"kubernetes.io/projected/e7850257-5351-4dc2-b64e-6e1f5ee927bc-kube-api-access-r5cq5\") pod \"test-curl-7464-139976271669600\" (UID: \"e7850257-5351-4dc2-b64e-6e1f5ee927bc\") " pod="opendatahub/test-curl-7464-139976271669600" Jul 02 19:42:43.278811 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.278779 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5cq5\" (UniqueName: \"kubernetes.io/projected/e7850257-5351-4dc2-b64e-6e1f5ee927bc-kube-api-access-r5cq5\") pod \"test-curl-7464-139976271669600\" (UID: \"e7850257-5351-4dc2-b64e-6e1f5ee927bc\") " pod="opendatahub/test-curl-7464-139976271669600" Jul 02 19:42:43.383562 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.383472 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669600" Jul 02 19:42:43.529700 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.529674 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139976271669600"] Jul 02 19:42:43.531117 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:42:43.531095 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7850257_5351_4dc2_b64e_6e1f5ee927bc.slice/crio-235e8381b5abaf26a57748b3ab384135666f81754e31dfa730c02bee2e0c77bf WatchSource:0}: Error finding container 235e8381b5abaf26a57748b3ab384135666f81754e31dfa730c02bee2e0c77bf: Status 404 returned error can't find the container with id 235e8381b5abaf26a57748b3ab384135666f81754e31dfa730c02bee2e0c77bf Jul 02 19:42:43.759005 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.758913 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139976271669600" event={"ID":"e7850257-5351-4dc2-b64e-6e1f5ee927bc","Type":"ContainerStarted","Data":"235e8381b5abaf26a57748b3ab384135666f81754e31dfa730c02bee2e0c77bf"} Jul 02 19:42:43.760213 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.760179 2571 scope.go:117] "RemoveContainer" containerID="2499acf24182be4da74394251981fd513d7a5413acf27071cfbe30cc44eeae12" Jul 02 19:42:43.760341 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:43.760228 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669488" Jul 02 19:42:44.768045 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:44.768017 2571 generic.go:356] "Generic (PLEG): container finished" podID="e7850257-5351-4dc2-b64e-6e1f5ee927bc" containerID="e283b85d46ef6577e038143ace241604fac84b7a7e0ba2bd5dab39b2036da22e" exitCode=0 Jul 02 19:42:44.768437 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:44.768108 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139976271669600" event={"ID":"e7850257-5351-4dc2-b64e-6e1f5ee927bc","Type":"ContainerDied","Data":"e283b85d46ef6577e038143ace241604fac84b7a7e0ba2bd5dab39b2036da22e"} Jul 02 19:42:45.915063 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:45.915034 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669600" Jul 02 19:42:45.934939 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:45.934910 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-139976271669600_e7850257-5351-4dc2-b64e-6e1f5ee927bc/test-curl-7464-139976271669600/0.log" Jul 02 19:42:45.966521 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:45.966476 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-139976271669600"] Jul 02 19:42:45.968914 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:45.968892 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-139976271669600"] Jul 02 19:42:45.997682 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:45.997654 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5cq5\" (UniqueName: \"kubernetes.io/projected/e7850257-5351-4dc2-b64e-6e1f5ee927bc-kube-api-access-r5cq5\") pod \"e7850257-5351-4dc2-b64e-6e1f5ee927bc\" (UID: \"e7850257-5351-4dc2-b64e-6e1f5ee927bc\") " Jul 02 19:42:45.999804 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:45.999776 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7850257-5351-4dc2-b64e-6e1f5ee927bc-kube-api-access-r5cq5" (OuterVolumeSpecName: "kube-api-access-r5cq5") pod "e7850257-5351-4dc2-b64e-6e1f5ee927bc" (UID: "e7850257-5351-4dc2-b64e-6e1f5ee927bc"). InnerVolumeSpecName "kube-api-access-r5cq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:46.090046 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.089958 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-139976271668928"] Jul 02 19:42:46.090470 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.090455 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7850257-5351-4dc2-b64e-6e1f5ee927bc" containerName="test-curl-7464-139976271669600" Jul 02 19:42:46.090511 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.090472 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7850257-5351-4dc2-b64e-6e1f5ee927bc" containerName="test-curl-7464-139976271669600" Jul 02 19:42:46.090570 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.090560 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7850257-5351-4dc2-b64e-6e1f5ee927bc" containerName="test-curl-7464-139976271669600" Jul 02 19:42:46.093697 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.093681 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271668928" Jul 02 19:42:46.098522 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.098498 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5cq5\" (UniqueName: \"kubernetes.io/projected/e7850257-5351-4dc2-b64e-6e1f5ee927bc-kube-api-access-r5cq5\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:42:46.101179 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.101059 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139976271668928"] Jul 02 19:42:46.199613 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.199574 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mzvk\" (UniqueName: \"kubernetes.io/projected/d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8-kube-api-access-8mzvk\") pod \"test-curl-7464-139976271668928\" (UID: \"d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8\") " pod="opendatahub/test-curl-7464-139976271668928" Jul 02 19:42:46.300264 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.300210 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mzvk\" (UniqueName: \"kubernetes.io/projected/d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8-kube-api-access-8mzvk\") pod \"test-curl-7464-139976271668928\" (UID: \"d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8\") " pod="opendatahub/test-curl-7464-139976271668928" Jul 02 19:42:46.309455 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.309426 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mzvk\" (UniqueName: \"kubernetes.io/projected/d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8-kube-api-access-8mzvk\") pod \"test-curl-7464-139976271668928\" (UID: \"d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8\") " pod="opendatahub/test-curl-7464-139976271668928" Jul 02 19:42:46.404242 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.404148 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271668928" Jul 02 19:42:46.541014 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.540988 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139976271668928"] Jul 02 19:42:46.542320 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:42:46.542296 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd479eaf2_3a07_4af3_bbf4_9ada0d8e0ff8.slice/crio-13a9d42a4ef0dba16ed4651c0c5ac205a7b8fddbc08ea9c6d956c89e4f887d4e WatchSource:0}: Error finding container 13a9d42a4ef0dba16ed4651c0c5ac205a7b8fddbc08ea9c6d956c89e4f887d4e: Status 404 returned error can't find the container with id 13a9d42a4ef0dba16ed4651c0c5ac205a7b8fddbc08ea9c6d956c89e4f887d4e Jul 02 19:42:46.782254 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.782222 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139976271668928" event={"ID":"d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8","Type":"ContainerStarted","Data":"13a9d42a4ef0dba16ed4651c0c5ac205a7b8fddbc08ea9c6d956c89e4f887d4e"} Jul 02 19:42:46.783364 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.783340 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271669600" Jul 02 19:42:46.783364 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:46.783356 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="235e8381b5abaf26a57748b3ab384135666f81754e31dfa730c02bee2e0c77bf" Jul 02 19:42:46.793623 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:42:46.793598 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7464-139976271669600\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="e7850257-5351-4dc2-b64e-6e1f5ee927bc" pod="opendatahub/test-curl-7464-139976271669600" Jul 02 19:42:47.238538 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:47.238455 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7850257-5351-4dc2-b64e-6e1f5ee927bc" path="/var/lib/kubelet/pods/e7850257-5351-4dc2-b64e-6e1f5ee927bc/volumes" Jul 02 19:42:47.790649 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:47.790617 2571 generic.go:356] "Generic (PLEG): container finished" podID="d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8" containerID="248ce1d86848a156b1ee4e6911aae558dd4eb4b7294b25cbde73cd1e729441a0" exitCode=0 Jul 02 19:42:47.790841 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:47.790706 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139976271668928" event={"ID":"d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8","Type":"ContainerDied","Data":"248ce1d86848a156b1ee4e6911aae558dd4eb4b7294b25cbde73cd1e729441a0"} Jul 02 19:42:48.959849 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:48.959823 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271668928" Jul 02 19:42:48.979388 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:48.979344 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-139976271668928_d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8/test-curl-7464-139976271668928/0.log" Jul 02 19:42:49.004331 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.004298 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-139976271668928"] Jul 02 19:42:49.008879 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.008855 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-139976271668928"] Jul 02 19:42:49.134314 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.134228 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mzvk\" (UniqueName: \"kubernetes.io/projected/d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8-kube-api-access-8mzvk\") pod \"d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8\" (UID: \"d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8\") " Jul 02 19:42:49.136293 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.136271 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8-kube-api-access-8mzvk" (OuterVolumeSpecName: "kube-api-access-8mzvk") pod "d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8" (UID: "d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8"). InnerVolumeSpecName "kube-api-access-8mzvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:49.173331 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.173301 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504"] Jul 02 19:42:49.173725 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.173713 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8" containerName="test-curl-7464-139976271668928" Jul 02 19:42:49.173770 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.173726 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8" containerName="test-curl-7464-139976271668928" Jul 02 19:42:49.173817 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.173808 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8" containerName="test-curl-7464-139976271668928" Jul 02 19:42:49.176790 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.176775 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504" Jul 02 19:42:49.179014 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.178990 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9629e8\"/\"openshift-service-ca.crt\"" Jul 02 19:42:49.179177 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.179143 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9629e8\"/\"kube-root-ca.crt\"" Jul 02 19:42:49.179815 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.179600 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9629e8\"/\"default-dockercfg-7bhmm\"" Jul 02 19:42:49.182536 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.182510 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504"] Jul 02 19:42:49.235232 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.235206 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mzvk\" (UniqueName: \"kubernetes.io/projected/d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8-kube-api-access-8mzvk\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:42:49.238095 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.238051 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8" path="/var/lib/kubelet/pods/d479eaf2-3a07-4af3-bbf4-9ada0d8e0ff8/volumes" Jul 02 19:42:49.336481 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.336436 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcr5g\" (UniqueName: \"kubernetes.io/projected/a6d1a865-0acb-431b-a6b1-644935e84710-kube-api-access-lcr5g\") pod \"test-curl-7464-139976269647504\" (UID: \"a6d1a865-0acb-431b-a6b1-644935e84710\") " pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504" Jul 02 19:42:49.437507 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.437395 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcr5g\" (UniqueName: \"kubernetes.io/projected/a6d1a865-0acb-431b-a6b1-644935e84710-kube-api-access-lcr5g\") pod \"test-curl-7464-139976269647504\" (UID: \"a6d1a865-0acb-431b-a6b1-644935e84710\") " pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504" Jul 02 19:42:49.450153 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.450127 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcr5g\" (UniqueName: \"kubernetes.io/projected/a6d1a865-0acb-431b-a6b1-644935e84710-kube-api-access-lcr5g\") pod \"test-curl-7464-139976269647504\" (UID: \"a6d1a865-0acb-431b-a6b1-644935e84710\") " pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504" Jul 02 19:42:49.486794 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.486760 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504" Jul 02 19:42:49.633224 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.633183 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504"] Jul 02 19:42:49.633604 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:42:49.633584 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6d1a865_0acb_431b_a6b1_644935e84710.slice/crio-9d268e209385546ff9a8f40bb49bc40a4b450be51119fab2e4bd85f2ec3ea43a WatchSource:0}: Error finding container 9d268e209385546ff9a8f40bb49bc40a4b450be51119fab2e4bd85f2ec3ea43a: Status 404 returned error can't find the container with id 9d268e209385546ff9a8f40bb49bc40a4b450be51119fab2e4bd85f2ec3ea43a Jul 02 19:42:49.802304 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.802279 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139976271668928" Jul 02 19:42:49.802478 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.802313 2571 scope.go:117] "RemoveContainer" containerID="248ce1d86848a156b1ee4e6911aae558dd4eb4b7294b25cbde73cd1e729441a0" Jul 02 19:42:49.803657 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:49.803620 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504" event={"ID":"a6d1a865-0acb-431b-a6b1-644935e84710","Type":"ContainerStarted","Data":"9d268e209385546ff9a8f40bb49bc40a4b450be51119fab2e4bd85f2ec3ea43a"} Jul 02 19:42:50.810241 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:50.810210 2571 generic.go:356] "Generic (PLEG): container finished" podID="a6d1a865-0acb-431b-a6b1-644935e84710" containerID="39310b453d1324d49eabf622e9c1d3663bff3f3626d8b7850505502c0214ee90" exitCode=6 Jul 02 19:42:50.810666 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:50.810304 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504" event={"ID":"a6d1a865-0acb-431b-a6b1-644935e84710","Type":"ContainerDied","Data":"39310b453d1324d49eabf622e9c1d3663bff3f3626d8b7850505502c0214ee90"} Jul 02 19:42:51.969062 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:51.969031 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504" Jul 02 19:42:51.989531 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:51.989507 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/ai-tenant-e2e-shared-a-9629e8_test-curl-7464-139976269647504_a6d1a865-0acb-431b-a6b1-644935e84710/test-curl-7464-139976269647504/0.log" Jul 02 19:42:52.031276 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.031240 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504"] Jul 02 19:42:52.034163 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.034138 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504"] Jul 02 19:42:52.066019 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.065985 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcr5g\" (UniqueName: \"kubernetes.io/projected/a6d1a865-0acb-431b-a6b1-644935e84710-kube-api-access-lcr5g\") pod \"a6d1a865-0acb-431b-a6b1-644935e84710\" (UID: \"a6d1a865-0acb-431b-a6b1-644935e84710\") " Jul 02 19:42:52.068251 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.068221 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6d1a865-0acb-431b-a6b1-644935e84710-kube-api-access-lcr5g" (OuterVolumeSpecName: "kube-api-access-lcr5g") pod "a6d1a865-0acb-431b-a6b1-644935e84710" (UID: "a6d1a865-0acb-431b-a6b1-644935e84710"). InnerVolumeSpecName "kube-api-access-lcr5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:52.167611 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.167577 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcr5g\" (UniqueName: \"kubernetes.io/projected/a6d1a865-0acb-431b-a6b1-644935e84710-kube-api-access-lcr5g\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:42:52.237636 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.237603 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984"] Jul 02 19:42:52.238232 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.238217 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6d1a865-0acb-431b-a6b1-644935e84710" containerName="test-curl-7464-139976269647504" Jul 02 19:42:52.238276 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.238235 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6d1a865-0acb-431b-a6b1-644935e84710" containerName="test-curl-7464-139976269647504" Jul 02 19:42:52.238357 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.238346 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6d1a865-0acb-431b-a6b1-644935e84710" containerName="test-curl-7464-139976269647504" Jul 02 19:42:52.241821 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.241804 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984" Jul 02 19:42:52.252437 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.252413 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984"] Jul 02 19:42:52.369923 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.369824 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lnsn\" (UniqueName: \"kubernetes.io/projected/ea054eef-1b0a-44d1-8165-b498533533a0-kube-api-access-4lnsn\") pod \"test-curl-7464-139976267391984\" (UID: \"ea054eef-1b0a-44d1-8165-b498533533a0\") " pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984" Jul 02 19:42:52.471320 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.471282 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lnsn\" (UniqueName: \"kubernetes.io/projected/ea054eef-1b0a-44d1-8165-b498533533a0-kube-api-access-4lnsn\") pod \"test-curl-7464-139976267391984\" (UID: \"ea054eef-1b0a-44d1-8165-b498533533a0\") " pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984" Jul 02 19:42:52.480602 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.480577 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lnsn\" (UniqueName: \"kubernetes.io/projected/ea054eef-1b0a-44d1-8165-b498533533a0-kube-api-access-4lnsn\") pod \"test-curl-7464-139976267391984\" (UID: \"ea054eef-1b0a-44d1-8165-b498533533a0\") " pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984" Jul 02 19:42:52.551809 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.551779 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984" Jul 02 19:42:52.689280 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.689257 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984"] Jul 02 19:42:52.690513 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:42:52.690489 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea054eef_1b0a_44d1_8165_b498533533a0.slice/crio-7387d97ee965c04b857c4fdb4a9b75979eec00782e83b708698f75015849c841 WatchSource:0}: Error finding container 7387d97ee965c04b857c4fdb4a9b75979eec00782e83b708698f75015849c841: Status 404 returned error can't find the container with id 7387d97ee965c04b857c4fdb4a9b75979eec00782e83b708698f75015849c841 Jul 02 19:42:52.826161 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.826130 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504" Jul 02 19:42:52.826353 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.826171 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d268e209385546ff9a8f40bb49bc40a4b450be51119fab2e4bd85f2ec3ea43a" Jul 02 19:42:52.827416 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:52.827387 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984" event={"ID":"ea054eef-1b0a-44d1-8165-b498533533a0","Type":"ContainerStarted","Data":"7387d97ee965c04b857c4fdb4a9b75979eec00782e83b708698f75015849c841"} Jul 02 19:42:52.836416 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:42:52.836389 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7464-139976269647504\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"ai-tenant-e2e-shared-a-9629e8\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="a6d1a865-0acb-431b-a6b1-644935e84710" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504" Jul 02 19:42:53.236168 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:42:53.236131 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7464-139976269647504\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"ai-tenant-e2e-shared-a-9629e8\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="a6d1a865-0acb-431b-a6b1-644935e84710" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976269647504" Jul 02 19:42:53.238138 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:53.238112 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6d1a865-0acb-431b-a6b1-644935e84710" path="/var/lib/kubelet/pods/a6d1a865-0acb-431b-a6b1-644935e84710/volumes" Jul 02 19:42:53.838169 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:53.838130 2571 generic.go:356] "Generic (PLEG): container finished" podID="ea054eef-1b0a-44d1-8165-b498533533a0" containerID="54af7068569384be51c15244c29b3593eec1451f190d4cab6ab401be4378dce4" exitCode=6 Jul 02 19:42:53.838339 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:53.838218 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984" event={"ID":"ea054eef-1b0a-44d1-8165-b498533533a0","Type":"ContainerDied","Data":"54af7068569384be51c15244c29b3593eec1451f190d4cab6ab401be4378dce4"} Jul 02 19:42:54.995310 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:54.995280 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984" Jul 02 19:42:54.995718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:54.995618 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lnsn\" (UniqueName: \"kubernetes.io/projected/ea054eef-1b0a-44d1-8165-b498533533a0-kube-api-access-4lnsn\") pod \"ea054eef-1b0a-44d1-8165-b498533533a0\" (UID: \"ea054eef-1b0a-44d1-8165-b498533533a0\") " Jul 02 19:42:54.997633 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:54.997601 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea054eef-1b0a-44d1-8165-b498533533a0-kube-api-access-4lnsn" (OuterVolumeSpecName: "kube-api-access-4lnsn") pod "ea054eef-1b0a-44d1-8165-b498533533a0" (UID: "ea054eef-1b0a-44d1-8165-b498533533a0"). InnerVolumeSpecName "kube-api-access-4lnsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:55.015945 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.015914 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/ai-tenant-e2e-shared-a-9629e8_test-curl-7464-139976267391984_ea054eef-1b0a-44d1-8165-b498533533a0/test-curl-7464-139976267391984/0.log" Jul 02 19:42:55.043040 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.043000 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984"] Jul 02 19:42:55.046465 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.046438 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984"] Jul 02 19:42:55.097138 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.097053 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lnsn\" (UniqueName: \"kubernetes.io/projected/ea054eef-1b0a-44d1-8165-b498533533a0-kube-api-access-4lnsn\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:42:55.191259 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.191224 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128"] Jul 02 19:42:55.191662 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.191648 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea054eef-1b0a-44d1-8165-b498533533a0" containerName="test-curl-7464-139976267391984" Jul 02 19:42:55.191728 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.191663 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea054eef-1b0a-44d1-8165-b498533533a0" containerName="test-curl-7464-139976267391984" Jul 02 19:42:55.191764 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.191738 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea054eef-1b0a-44d1-8165-b498533533a0" containerName="test-curl-7464-139976267391984" Jul 02 19:42:55.194732 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.194717 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128" Jul 02 19:42:55.197632 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.197604 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9pdp\" (UniqueName: \"kubernetes.io/projected/8d1bc1d1-c592-4d3f-9800-f666c9b55898-kube-api-access-z9pdp\") pod \"test-curl-7464-139976270542128\" (UID: \"8d1bc1d1-c592-4d3f-9800-f666c9b55898\") " pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128" Jul 02 19:42:55.200599 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.200578 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128"] Jul 02 19:42:55.238415 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.238383 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea054eef-1b0a-44d1-8165-b498533533a0" path="/var/lib/kubelet/pods/ea054eef-1b0a-44d1-8165-b498533533a0/volumes" Jul 02 19:42:55.298248 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.298211 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9pdp\" (UniqueName: \"kubernetes.io/projected/8d1bc1d1-c592-4d3f-9800-f666c9b55898-kube-api-access-z9pdp\") pod \"test-curl-7464-139976270542128\" (UID: \"8d1bc1d1-c592-4d3f-9800-f666c9b55898\") " pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128" Jul 02 19:42:55.306503 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.306473 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9pdp\" (UniqueName: \"kubernetes.io/projected/8d1bc1d1-c592-4d3f-9800-f666c9b55898-kube-api-access-z9pdp\") pod \"test-curl-7464-139976270542128\" (UID: \"8d1bc1d1-c592-4d3f-9800-f666c9b55898\") " pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128" Jul 02 19:42:55.504720 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.504632 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128" Jul 02 19:42:55.647852 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.647829 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128"] Jul 02 19:42:55.649907 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:42:55.649876 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d1bc1d1_c592_4d3f_9800_f666c9b55898.slice/crio-af1b5d65b2c6b83ee6967b51ae2c4f4ba838d142323ab1f8efa7730661c9bea6 WatchSource:0}: Error finding container af1b5d65b2c6b83ee6967b51ae2c4f4ba838d142323ab1f8efa7730661c9bea6: Status 404 returned error can't find the container with id af1b5d65b2c6b83ee6967b51ae2c4f4ba838d142323ab1f8efa7730661c9bea6 Jul 02 19:42:55.852363 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.852329 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128" event={"ID":"8d1bc1d1-c592-4d3f-9800-f666c9b55898","Type":"ContainerStarted","Data":"af1b5d65b2c6b83ee6967b51ae2c4f4ba838d142323ab1f8efa7730661c9bea6"} Jul 02 19:42:55.853617 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.853598 2571 scope.go:117] "RemoveContainer" containerID="54af7068569384be51c15244c29b3593eec1451f190d4cab6ab401be4378dce4" Jul 02 19:42:55.853617 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:55.853614 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976267391984" Jul 02 19:42:56.862059 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:56.862023 2571 generic.go:356] "Generic (PLEG): container finished" podID="8d1bc1d1-c592-4d3f-9800-f666c9b55898" containerID="55445b230ee64a5e733dda4bd3880b6addd24c339f85b8f89cd20321f1976139" exitCode=6 Jul 02 19:42:56.862523 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:56.862123 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128" event={"ID":"8d1bc1d1-c592-4d3f-9800-f666c9b55898","Type":"ContainerDied","Data":"55445b230ee64a5e733dda4bd3880b6addd24c339f85b8f89cd20321f1976139"} Jul 02 19:42:58.013377 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:58.013323 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128" Jul 02 19:42:58.023800 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:58.023772 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9pdp\" (UniqueName: \"kubernetes.io/projected/8d1bc1d1-c592-4d3f-9800-f666c9b55898-kube-api-access-z9pdp\") pod \"8d1bc1d1-c592-4d3f-9800-f666c9b55898\" (UID: \"8d1bc1d1-c592-4d3f-9800-f666c9b55898\") " Jul 02 19:42:58.026162 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:58.026137 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d1bc1d1-c592-4d3f-9800-f666c9b55898-kube-api-access-z9pdp" (OuterVolumeSpecName: "kube-api-access-z9pdp") pod "8d1bc1d1-c592-4d3f-9800-f666c9b55898" (UID: "8d1bc1d1-c592-4d3f-9800-f666c9b55898"). InnerVolumeSpecName "kube-api-access-z9pdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:42:58.036784 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:58.036762 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/ai-tenant-e2e-shared-a-9629e8_test-curl-7464-139976270542128_8d1bc1d1-c592-4d3f-9800-f666c9b55898/test-curl-7464-139976270542128/0.log" Jul 02 19:42:58.065470 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:58.065439 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128"] Jul 02 19:42:58.069741 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:58.069713 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128"] Jul 02 19:42:58.124897 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:58.124795 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9pdp\" (UniqueName: \"kubernetes.io/projected/8d1bc1d1-c592-4d3f-9800-f666c9b55898-kube-api-access-z9pdp\") on node \"ip-10-0-138-101.ec2.internal\" DevicePath \"\"" Jul 02 19:42:58.876212 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:58.876183 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af1b5d65b2c6b83ee6967b51ae2c4f4ba838d142323ab1f8efa7730661c9bea6" Jul 02 19:42:58.876212 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:58.876188 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128" Jul 02 19:42:58.886972 ip-10-0-138-101 kubenswrapper[2571]: E0702 19:42:58.886938 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7464-139976270542128\" is forbidden: User \"system:node:ip-10-0-138-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"ai-tenant-e2e-shared-a-9629e8\": no relationship found between node 'ip-10-0-138-101.ec2.internal' and this object" podUID="8d1bc1d1-c592-4d3f-9800-f666c9b55898" pod="ai-tenant-e2e-shared-a-9629e8/test-curl-7464-139976270542128" Jul 02 19:42:59.238218 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:42:59.238140 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d1bc1d1-c592-4d3f-9800-f666c9b55898" path="/var/lib/kubelet/pods/8d1bc1d1-c592-4d3f-9800-f666c9b55898/volumes" Jul 02 19:43:12.982941 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:12.982905 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-845f95f44b-dmj2w_1af8b5a9-e426-4a3c-9697-df2256ab0e5c/authorino/0.log" Jul 02 19:43:17.577971 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:17.577933 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-49bv6_ad9662ae-db56-40a3-95dd-f1768572d712/manager/0.log" Jul 02 19:43:17.698878 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:17.698850 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jvv9m_128fa7ca-3973-4884-830e-5b88f8bf187e/manager/0.log" Jul 02 19:43:17.821837 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:17.821805 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-9dcfd9c48-lgrqf_477b13c6-d88e-4e12-8fdc-ee7395f4c179/maas-api/0.log" Jul 02 19:43:18.177715 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:18.177685 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-85fc68f447-vjvhf_6ba6d0e2-f568-4e1e-abb9-415c27a00439/manager/0.log" Jul 02 19:43:18.291445 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:18.291421 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9jv86_83fa00f0-2ff4-47c3-9af9-f1deb313a786/server/0.log" Jul 02 19:43:18.410494 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:18.410467 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-szvmv_51107ee4-db1f-4dfc-b67c-f43269462379/manager/0.log" Jul 02 19:43:18.771758 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:18.771731 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd5847cc4-x9xwf_1b65de0f-2d80-4220-8186-a77feb3f847e/copy-manifests/0.log" Jul 02 19:43:18.785294 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:18.785261 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd5847cc4-x9xwf_1b65de0f-2d80-4220-8186-a77feb3f847e/manager/0.log" Jul 02 19:43:18.905536 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:18.905511 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-xzbfb_570c0902-be34-4482-b33f-71ecf681f742/postgres/0.log" Jul 02 19:43:19.674302 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:19.674270 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz_164cd298-6237-4212-aa0b-cb6004c464f2/util/0.log" Jul 02 19:43:19.681988 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:19.681962 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz_164cd298-6237-4212-aa0b-cb6004c464f2/pull/0.log" Jul 02 19:43:19.688603 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:19.688584 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz_164cd298-6237-4212-aa0b-cb6004c464f2/extract/0.log" Jul 02 19:43:19.803121 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:19.803097 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm_e6d8b20e-5192-482a-af1f-33e3ba49ec74/util/0.log" Jul 02 19:43:19.809967 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:19.809944 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm_e6d8b20e-5192-482a-af1f-33e3ba49ec74/pull/0.log" Jul 02 19:43:19.816934 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:19.816913 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm_e6d8b20e-5192-482a-af1f-33e3ba49ec74/extract/0.log" Jul 02 19:43:19.929133 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:19.929023 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5_a378be6c-22a8-419d-b887-2619dfb8024f/util/0.log" Jul 02 19:43:19.936187 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:19.936162 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5_a378be6c-22a8-419d-b887-2619dfb8024f/pull/0.log" Jul 02 19:43:19.943268 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:19.943247 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5_a378be6c-22a8-419d-b887-2619dfb8024f/extract/0.log" Jul 02 19:43:20.054127 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:20.054096 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh_80e2a77b-0851-4bc5-8ed0-c6d49ec62683/util/0.log" Jul 02 19:43:20.061487 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:20.061459 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh_80e2a77b-0851-4bc5-8ed0-c6d49ec62683/pull/0.log" Jul 02 19:43:20.068271 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:20.068250 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh_80e2a77b-0851-4bc5-8ed0-c6d49ec62683/extract/0.log" Jul 02 19:43:20.186861 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:20.186778 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-845f95f44b-dmj2w_1af8b5a9-e426-4a3c-9697-df2256ab0e5c/authorino/0.log" Jul 02 19:43:20.309513 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:20.309487 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6xvfk_e874b86d-4fd3-48da-9d47-474a6bc99de4/manager/0.log" Jul 02 19:43:20.423664 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:20.423636 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qb9bf_52da5f25-6026-4f0f-829c-30d22e41ccc4/manager/0.log" Jul 02 19:43:20.536399 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:20.536376 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-b5jp8_0acce477-a26f-4073-b651-8bcfc6b7cbc9/kuadrant-console-plugin/0.log" Jul 02 19:43:20.653860 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:20.653830 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6f828_1c6b86a0-461b-4bec-a4ae-b63d652580b9/registry-server/0.log" Jul 02 19:43:21.004661 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:21.004578 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9r9fj_3f0232b3-b880-4da1-9b04-6d1c62e637de/manager/0.log" Jul 02 19:43:21.706617 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:21.706590 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7656b89f9d-ccn4t_7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5/kube-auth-proxy/0.log" Jul 02 19:43:21.824199 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:21.824170 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bqr69_d2515057-7616-44fa-8b3c-5878008819af/istio-proxy/1.log" Jul 02 19:43:21.953449 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:21.953419 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-nzb5l_2955ab2a-8039-463f-8d59-52950e3afadd/payload-pre-processing/0.log" Jul 02 19:43:22.069713 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:22.069682 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gzg82_a6ec0e33-fb2e-4756-83e7-8fec1176a82f/payload-processing/0.log" Jul 02 19:43:22.535946 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:22.535920 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-f8g6m_247553b6-f84b-4f02-a7ef-5b1f99909113/main/0.log" Jul 02 19:43:22.646237 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:22.646208 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-lx9zc_ea4bc6a9-7565-4a99-b702-fdba2141ba9e/main/0.log" Jul 02 19:43:23.100383 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:23.100354 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-q6czb_e0d2b639-8daf-4887-b3aa-f3529242831f/main/0.log" Jul 02 19:43:27.480239 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.480207 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d52kh/must-gather-jntwm"] Jul 02 19:43:27.480671 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.480658 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d1bc1d1-c592-4d3f-9800-f666c9b55898" containerName="test-curl-7464-139976270542128" Jul 02 19:43:27.480671 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.480673 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d1bc1d1-c592-4d3f-9800-f666c9b55898" containerName="test-curl-7464-139976270542128" Jul 02 19:43:27.480750 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.480739 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d1bc1d1-c592-4d3f-9800-f666c9b55898" containerName="test-curl-7464-139976270542128" Jul 02 19:43:27.484333 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.484306 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d52kh/must-gather-jntwm" Jul 02 19:43:27.489192 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.489172 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d52kh\"/\"kube-root-ca.crt\"" Jul 02 19:43:27.489945 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.489923 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d52kh\"/\"default-dockercfg-n4n8v\"" Jul 02 19:43:27.490062 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.489924 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d52kh\"/\"openshift-service-ca.crt\"" Jul 02 19:43:27.498614 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.498588 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d52kh/must-gather-jntwm"] Jul 02 19:43:27.608623 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.608592 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwpkh\" (UniqueName: \"kubernetes.io/projected/6a24d29b-6483-47b9-a79f-18166d8434f6-kube-api-access-xwpkh\") pod \"must-gather-jntwm\" (UID: \"6a24d29b-6483-47b9-a79f-18166d8434f6\") " pod="openshift-must-gather-d52kh/must-gather-jntwm" Jul 02 19:43:27.608807 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.608631 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6a24d29b-6483-47b9-a79f-18166d8434f6-must-gather-output\") pod \"must-gather-jntwm\" (UID: \"6a24d29b-6483-47b9-a79f-18166d8434f6\") " pod="openshift-must-gather-d52kh/must-gather-jntwm" Jul 02 19:43:27.709202 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.709162 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwpkh\" (UniqueName: \"kubernetes.io/projected/6a24d29b-6483-47b9-a79f-18166d8434f6-kube-api-access-xwpkh\") pod \"must-gather-jntwm\" (UID: \"6a24d29b-6483-47b9-a79f-18166d8434f6\") " pod="openshift-must-gather-d52kh/must-gather-jntwm" Jul 02 19:43:27.709202 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.709206 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6a24d29b-6483-47b9-a79f-18166d8434f6-must-gather-output\") pod \"must-gather-jntwm\" (UID: \"6a24d29b-6483-47b9-a79f-18166d8434f6\") " pod="openshift-must-gather-d52kh/must-gather-jntwm" Jul 02 19:43:27.709589 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.709573 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6a24d29b-6483-47b9-a79f-18166d8434f6-must-gather-output\") pod \"must-gather-jntwm\" (UID: \"6a24d29b-6483-47b9-a79f-18166d8434f6\") " pod="openshift-must-gather-d52kh/must-gather-jntwm" Jul 02 19:43:27.717950 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.717920 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwpkh\" (UniqueName: \"kubernetes.io/projected/6a24d29b-6483-47b9-a79f-18166d8434f6-kube-api-access-xwpkh\") pod \"must-gather-jntwm\" (UID: \"6a24d29b-6483-47b9-a79f-18166d8434f6\") " pod="openshift-must-gather-d52kh/must-gather-jntwm" Jul 02 19:43:27.793894 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.793855 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d52kh/must-gather-jntwm" Jul 02 19:43:27.939294 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:27.939267 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d52kh/must-gather-jntwm"] Jul 02 19:43:27.941140 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:43:27.941115 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a24d29b_6483_47b9_a79f_18166d8434f6.slice/crio-2a8a9b775c6f88f663da82b1e2d602a95587ac06cbefc88aa2d97c2b7b46989a WatchSource:0}: Error finding container 2a8a9b775c6f88f663da82b1e2d602a95587ac06cbefc88aa2d97c2b7b46989a: Status 404 returned error can't find the container with id 2a8a9b775c6f88f663da82b1e2d602a95587ac06cbefc88aa2d97c2b7b46989a Jul 02 19:43:28.050384 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:28.050298 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d52kh/must-gather-jntwm" event={"ID":"6a24d29b-6483-47b9-a79f-18166d8434f6","Type":"ContainerStarted","Data":"2a8a9b775c6f88f663da82b1e2d602a95587ac06cbefc88aa2d97c2b7b46989a"} Jul 02 19:43:29.061053 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:29.061021 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d52kh/must-gather-jntwm" event={"ID":"6a24d29b-6483-47b9-a79f-18166d8434f6","Type":"ContainerStarted","Data":"3d9f27e4cde3cc86302c297bb1586076e394ead40c0245470a27a4eca49e5e74"} Jul 02 19:43:29.061053 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:29.061064 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d52kh/must-gather-jntwm" event={"ID":"6a24d29b-6483-47b9-a79f-18166d8434f6","Type":"ContainerStarted","Data":"d808ad32c8072ac51f911b0280bc2ccd8c41fd63f2a87bf8ea464306256c6be2"} Jul 02 19:43:29.080106 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:29.080022 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-d52kh/must-gather-jntwm" podStartSLOduration=1.380485768 podStartE2EDuration="2.080000782s" totalImagesPullingTime="699.515014ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:43:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:43:29.075203431 +0000 UTC m=+2446.366540330" watchObservedRunningTime="2026-07-02 19:43:29.080000782 +0000 UTC m=+2446.371337683" Jul 02 19:43:30.213061 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:30.213030 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8sprk_9332d088-8ecc-4c56-a0d8-6d35d5acda58/global-pull-secret-syncer/0.log" Jul 02 19:43:30.434041 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:30.433981 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kkggr_eee2ced5-efa4-498e-83f9-793366d871b7/konnectivity-agent/0.log" Jul 02 19:43:30.484498 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:30.484426 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-101.ec2.internal_9b6c8b6d42e5b52ab4cf9f706a9be0f7/haproxy/0.log" Jul 02 19:43:34.907557 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:34.907471 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz_164cd298-6237-4212-aa0b-cb6004c464f2/extract/0.log" Jul 02 19:43:34.941718 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:34.941647 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz_164cd298-6237-4212-aa0b-cb6004c464f2/util/0.log" Jul 02 19:43:34.997601 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:34.997454 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zfnmz_164cd298-6237-4212-aa0b-cb6004c464f2/pull/0.log" Jul 02 19:43:35.057979 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.057948 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm_e6d8b20e-5192-482a-af1f-33e3ba49ec74/extract/0.log" Jul 02 19:43:35.127822 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.127797 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm_e6d8b20e-5192-482a-af1f-33e3ba49ec74/util/0.log" Jul 02 19:43:35.153976 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.153949 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lkgtm_e6d8b20e-5192-482a-af1f-33e3ba49ec74/pull/0.log" Jul 02 19:43:35.188958 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.188873 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5_a378be6c-22a8-419d-b887-2619dfb8024f/extract/0.log" Jul 02 19:43:35.217961 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.217937 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5_a378be6c-22a8-419d-b887-2619dfb8024f/util/0.log" Jul 02 19:43:35.251753 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.251721 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732v8n5_a378be6c-22a8-419d-b887-2619dfb8024f/pull/0.log" Jul 02 19:43:35.286237 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.286202 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh_80e2a77b-0851-4bc5-8ed0-c6d49ec62683/extract/0.log" Jul 02 19:43:35.310743 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.310708 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh_80e2a77b-0851-4bc5-8ed0-c6d49ec62683/util/0.log" Jul 02 19:43:35.334991 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.334910 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hdhsh_80e2a77b-0851-4bc5-8ed0-c6d49ec62683/pull/0.log" Jul 02 19:43:35.693092 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.692055 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-845f95f44b-dmj2w_1af8b5a9-e426-4a3c-9697-df2256ab0e5c/authorino/0.log" Jul 02 19:43:35.737555 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.737528 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6xvfk_e874b86d-4fd3-48da-9d47-474a6bc99de4/manager/0.log" Jul 02 19:43:35.763231 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.763199 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qb9bf_52da5f25-6026-4f0f-829c-30d22e41ccc4/manager/0.log" Jul 02 19:43:35.791418 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.791394 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-b5jp8_0acce477-a26f-4073-b651-8bcfc6b7cbc9/kuadrant-console-plugin/0.log" Jul 02 19:43:35.833663 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:35.833622 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6f828_1c6b86a0-461b-4bec-a4ae-b63d652580b9/registry-server/0.log" Jul 02 19:43:36.078696 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:36.078533 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9r9fj_3f0232b3-b880-4da1-9b04-6d1c62e637de/manager/0.log" Jul 02 19:43:37.750314 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:37.750233 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_241b3746-c7e5-43d4-9e38-cc0b2d7a9b50/alertmanager/0.log" Jul 02 19:43:37.779450 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:37.779405 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_241b3746-c7e5-43d4-9e38-cc0b2d7a9b50/config-reloader/0.log" Jul 02 19:43:37.799629 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:37.799597 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_241b3746-c7e5-43d4-9e38-cc0b2d7a9b50/kube-rbac-proxy-web/0.log" Jul 02 19:43:37.822439 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:37.822408 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_241b3746-c7e5-43d4-9e38-cc0b2d7a9b50/kube-rbac-proxy/0.log" Jul 02 19:43:37.849461 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:37.849432 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_241b3746-c7e5-43d4-9e38-cc0b2d7a9b50/kube-rbac-proxy-metric/0.log" Jul 02 19:43:37.873732 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:37.873699 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_241b3746-c7e5-43d4-9e38-cc0b2d7a9b50/prom-label-proxy/0.log" Jul 02 19:43:37.902838 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:37.902804 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_241b3746-c7e5-43d4-9e38-cc0b2d7a9b50/init-config-reloader/0.log" Jul 02 19:43:37.976228 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:37.976194 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-g9lwm_96f57550-6488-4662-ad6c-0241cd5035f1/kube-state-metrics/0.log" Jul 02 19:43:37.993043 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:37.993017 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-g9lwm_96f57550-6488-4662-ad6c-0241cd5035f1/kube-rbac-proxy-main/0.log" Jul 02 19:43:38.015348 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:38.015316 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-g9lwm_96f57550-6488-4662-ad6c-0241cd5035f1/kube-rbac-proxy-self/0.log" Jul 02 19:43:38.047705 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:38.047665 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5bd5947889-lfh8v_1d4e7167-ea2a-4cb7-9c30-b29a87cb07f3/metrics-server/0.log" Jul 02 19:43:38.075962 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:38.075941 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-bp5hj_a75a2281-4146-464f-bdda-8f221a98f52d/monitoring-plugin/0.log" Jul 02 19:43:38.108422 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:38.108391 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9gmrb_a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4/node-exporter/0.log" Jul 02 19:43:38.134861 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:38.134830 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9gmrb_a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4/kube-rbac-proxy/0.log" Jul 02 19:43:38.166403 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:38.166375 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9gmrb_a89c3e35-f2b0-4d44-8e34-7b23f6cb05f4/init-textfile/0.log" Jul 02 19:43:38.620517 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:38.620478 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-h7hwg_34484f96-7f73-406e-af29-d1e7621e4580/prometheus-operator-admission-webhook/0.log" Jul 02 19:43:38.649642 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:38.649614 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-785858fb6-bvr9s_10732671-4620-4659-a188-71fefbcf5333/telemeter-client/0.log" Jul 02 19:43:38.672236 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:38.672207 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-785858fb6-bvr9s_10732671-4620-4659-a188-71fefbcf5333/reload/0.log" Jul 02 19:43:38.695685 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:38.695648 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-785858fb6-bvr9s_10732671-4620-4659-a188-71fefbcf5333/kube-rbac-proxy/0.log" Jul 02 19:43:40.474274 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.474234 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k"] Jul 02 19:43:40.483005 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.482967 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.488493 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.488464 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k"] Jul 02 19:43:40.566133 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.566064 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-proc\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.566623 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.566562 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-podres\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.566787 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.566770 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvf8g\" (UniqueName: \"kubernetes.io/projected/5a997935-ccc1-48ec-8841-9414c2b216d8-kube-api-access-rvf8g\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.566914 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.566901 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-sys\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.567191 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.567176 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-lib-modules\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.668254 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.668214 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-proc\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.668478 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.668444 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-proc\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.668620 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.668599 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-podres\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.668760 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.668742 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvf8g\" (UniqueName: \"kubernetes.io/projected/5a997935-ccc1-48ec-8841-9414c2b216d8-kube-api-access-rvf8g\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.668886 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.668870 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-sys\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.669147 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.669127 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-lib-modules\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.669442 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.669424 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-lib-modules\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.669641 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.669624 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-podres\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.670109 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.670088 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a997935-ccc1-48ec-8841-9414c2b216d8-sys\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.680624 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.680589 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvf8g\" (UniqueName: \"kubernetes.io/projected/5a997935-ccc1-48ec-8841-9414c2b216d8-kube-api-access-rvf8g\") pod \"perf-node-gather-daemonset-2nz5k\" (UID: \"5a997935-ccc1-48ec-8841-9414c2b216d8\") " pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:40.805866 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:40.805831 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:41.003838 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:41.003807 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k"] Jul 02 19:43:41.005506 ip-10-0-138-101 kubenswrapper[2571]: W0702 19:43:41.005469 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5a997935_ccc1_48ec_8841_9414c2b216d8.slice/crio-e617f13db7fcfa0f13e7d484a5526b55843b4d5117ef66f644be8f8d70aaaa4d WatchSource:0}: Error finding container e617f13db7fcfa0f13e7d484a5526b55843b4d5117ef66f644be8f8d70aaaa4d: Status 404 returned error can't find the container with id e617f13db7fcfa0f13e7d484a5526b55843b4d5117ef66f644be8f8d70aaaa4d Jul 02 19:43:41.182524 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:41.182482 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" event={"ID":"5a997935-ccc1-48ec-8841-9414c2b216d8","Type":"ContainerStarted","Data":"d91ac2f6817ebc71499cc1221f6861ed05b0ccec1c07f24c021012076f1d01c8"} Jul 02 19:43:41.182721 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:41.182691 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:41.182792 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:41.182731 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" event={"ID":"5a997935-ccc1-48ec-8841-9414c2b216d8","Type":"ContainerStarted","Data":"e617f13db7fcfa0f13e7d484a5526b55843b4d5117ef66f644be8f8d70aaaa4d"} Jul 02 19:43:41.200311 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:41.200259 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" podStartSLOduration=1.2002422990000001 podStartE2EDuration="1.200242299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:43:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:43:41.197858029 +0000 UTC m=+2458.489194934" watchObservedRunningTime="2026-07-02 19:43:41.200242299 +0000 UTC m=+2458.491579197" Jul 02 19:43:43.004307 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:43.004274 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c2szs_9309e0d2-176b-4f90-bb3c-fc117cdea733/dns/0.log" Jul 02 19:43:43.045088 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:43.045046 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c2szs_9309e0d2-176b-4f90-bb3c-fc117cdea733/kube-rbac-proxy/0.log" Jul 02 19:43:43.220496 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:43.220466 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v2xxk_fe8227ad-efbc-48d6-bf5b-21902eeeb40f/dns-node-resolver/0.log" Jul 02 19:43:43.802232 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:43.802203 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c7bdd876f-4rccg_1da140cc-562a-4c88-b899-3001afc4e989/registry/0.log" Jul 02 19:43:43.849194 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:43.849166 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rv7zv_3eb8b0ed-ff24-4cdb-8bdc-288cc6a8aced/node-ca/0.log" Jul 02 19:43:45.017490 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:45.017457 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7656b89f9d-ccn4t_7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5/kube-auth-proxy/0.log" Jul 02 19:43:45.049477 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:45.049450 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bqr69_d2515057-7616-44fa-8b3c-5878008819af/istio-proxy/0.log" Jul 02 19:43:45.086567 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:45.086538 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bqr69_d2515057-7616-44fa-8b3c-5878008819af/istio-proxy/1.log" Jul 02 19:43:45.136648 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:45.136578 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-nzb5l_2955ab2a-8039-463f-8d59-52950e3afadd/payload-pre-processing/0.log" Jul 02 19:43:45.198916 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:45.198885 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gzg82_a6ec0e33-fb2e-4756-83e7-8fec1176a82f/payload-processing/0.log" Jul 02 19:43:45.883992 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:45.883964 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mckjl_aa480a02-4284-406b-b0a4-b72a2fc9800d/serve-healthcheck-canary/0.log" Jul 02 19:43:46.560287 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:46.560256 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pqz7k_5f92e3f7-20e6-440b-ad6a-b5fb175b19a3/kube-rbac-proxy/0.log" Jul 02 19:43:46.581104 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:46.581081 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pqz7k_5f92e3f7-20e6-440b-ad6a-b5fb175b19a3/exporter/0.log" Jul 02 19:43:46.603380 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:46.603349 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pqz7k_5f92e3f7-20e6-440b-ad6a-b5fb175b19a3/extractor/0.log" Jul 02 19:43:47.202317 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:47.202289 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d52kh/perf-node-gather-daemonset-2nz5k" Jul 02 19:43:48.910934 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:48.910907 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-49bv6_ad9662ae-db56-40a3-95dd-f1768572d712/manager/0.log" Jul 02 19:43:49.087408 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:49.087367 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jvv9m_128fa7ca-3973-4884-830e-5b88f8bf187e/manager/0.log" Jul 02 19:43:49.130977 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:49.130945 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-9dcfd9c48-lgrqf_477b13c6-d88e-4e12-8fdc-ee7395f4c179/maas-api/0.log" Jul 02 19:43:49.307922 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:49.307895 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-85fc68f447-vjvhf_6ba6d0e2-f568-4e1e-abb9-415c27a00439/manager/0.log" Jul 02 19:43:49.326905 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:49.326880 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9jv86_83fa00f0-2ff4-47c3-9af9-f1deb313a786/server/0.log" Jul 02 19:43:49.410088 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:49.410048 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-szvmv_51107ee4-db1f-4dfc-b67c-f43269462379/manager/0.log" Jul 02 19:43:49.602560 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:49.602483 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd5847cc4-x9xwf_1b65de0f-2d80-4220-8186-a77feb3f847e/manager/0.log" Jul 02 19:43:49.622373 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:49.622347 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd5847cc4-x9xwf_1b65de0f-2d80-4220-8186-a77feb3f847e/copy-manifests/0.log" Jul 02 19:43:49.651032 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:49.651001 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-xzbfb_570c0902-be34-4482-b33f-71ecf681f742/postgres/0.log" Jul 02 19:43:51.561061 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:51.561033 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7656b89f9d-ccn4t_7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5/kube-auth-proxy/0.log" Jul 02 19:43:51.580962 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:51.580928 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bqr69_d2515057-7616-44fa-8b3c-5878008819af/istio-proxy/0.log" Jul 02 19:43:51.617896 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:51.617866 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bqr69_d2515057-7616-44fa-8b3c-5878008819af/istio-proxy/1.log" Jul 02 19:43:51.655341 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:51.655306 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-nzb5l_2955ab2a-8039-463f-8d59-52950e3afadd/payload-pre-processing/0.log" Jul 02 19:43:51.705413 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:51.705382 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gzg82_a6ec0e33-fb2e-4756-83e7-8fec1176a82f/payload-processing/0.log" Jul 02 19:43:52.973678 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:52.973615 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7656b89f9d-ccn4t_7657fe49-4f8a-4f02-a0b7-5bc7ced7c1c5/kube-auth-proxy/0.log" Jul 02 19:43:52.997994 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:52.997498 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bqr69_d2515057-7616-44fa-8b3c-5878008819af/istio-proxy/0.log" Jul 02 19:43:53.042303 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:53.042263 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bqr69_d2515057-7616-44fa-8b3c-5878008819af/istio-proxy/1.log" Jul 02 19:43:53.079805 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:53.079771 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-nzb5l_2955ab2a-8039-463f-8d59-52950e3afadd/payload-pre-processing/0.log" Jul 02 19:43:53.128689 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:53.128660 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gzg82_a6ec0e33-fb2e-4756-83e7-8fec1176a82f/payload-processing/0.log" Jul 02 19:43:57.456553 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:57.456525 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7tb99_45980b15-071c-4aa7-a6e6-8ae7c9d48de2/kube-multus/0.log" Jul 02 19:43:57.486266 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:57.486236 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z9qb_cb60b6fe-625d-4029-9082-cdd088ab2179/kube-multus-additional-cni-plugins/0.log" Jul 02 19:43:57.510410 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:57.510382 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z9qb_cb60b6fe-625d-4029-9082-cdd088ab2179/egress-router-binary-copy/0.log" Jul 02 19:43:57.532468 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:57.532443 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z9qb_cb60b6fe-625d-4029-9082-cdd088ab2179/cni-plugins/0.log" Jul 02 19:43:57.553578 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:57.553555 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z9qb_cb60b6fe-625d-4029-9082-cdd088ab2179/bond-cni-plugin/0.log" Jul 02 19:43:57.582250 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:57.582224 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z9qb_cb60b6fe-625d-4029-9082-cdd088ab2179/routeoverride-cni/0.log" Jul 02 19:43:57.607876 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:57.607838 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z9qb_cb60b6fe-625d-4029-9082-cdd088ab2179/whereabouts-cni-bincopy/0.log" Jul 02 19:43:57.628815 ip-10-0-138-101 kubenswrapper[2571]: I0702 19:43:57.628787 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9z9qb_cb60b6fe-625d-4029-9082-cdd088ab2179/whereabouts-cni/0.log"