Aug 04 09:38:25.219985 ip-10-0-143-10 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Aug 04 09:38:25.219995 ip-10-0-143-10 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Aug 04 09:38:25.220002 ip-10-0-143-10 systemd[1]: kubelet.service: Failed with result 'resources'. Aug 04 09:38:25.220236 ip-10-0-143-10 systemd[1]: Failed to start Kubernetes Kubelet. Aug 04 09:38:35.341994 ip-10-0-143-10 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Aug 04 09:38:35.342014 ip-10-0-143-10 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 30cf3d12c8ac4d9585244a734ec95d83 -- Aug 04 09:40:29.503622 ip-10-0-143-10 systemd[1]: Starting Kubernetes Kubelet... Aug 04 09:40:29.858876 ip-10-0-143-10 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 09:40:29.858876 ip-10-0-143-10 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 04 09:40:29.858876 ip-10-0-143-10 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 09:40:29.858876 ip-10-0-143-10 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 04 09:40:29.858876 ip-10-0-143-10 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 09:40:29.859832 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.859737 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 04 09:40:29.866096 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866069 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:29.866096 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866092 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:29.866096 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866096 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:29.866096 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866099 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:29.866096 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866104 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866110 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866113 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866117 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866120 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866123 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866126 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866128 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866131 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866133 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866135 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866138 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866141 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866143 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866146 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866148 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866151 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866154 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866157 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:29.866326 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866159 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866162 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866165 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866168 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866171 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866173 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866175 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866178 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866181 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866183 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866185 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866188 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866190 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866193 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866196 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866199 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866202 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866205 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866207 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:29.866768 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866210 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866213 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866215 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866218 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866220 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866223 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866225 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866229 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866232 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866235 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866237 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866240 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866242 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866245 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866263 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866266 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866268 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866271 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866273 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866276 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:29.867234 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866278 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866281 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866283 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866286 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866288 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866291 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866293 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866296 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866299 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866302 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866305 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866308 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866310 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866313 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866316 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866318 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866321 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866323 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866326 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866329 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:29.867737 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866331 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866336 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866338 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866342 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866346 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866350 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866353 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866356 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866359 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866363 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866367 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866370 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866373 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866376 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866378 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866381 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.866384 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867110 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:29.868192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867118 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867122 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867126 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867130 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867133 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867136 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867139 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867142 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867145 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867147 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867151 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867153 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867156 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867159 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867161 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867164 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867167 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867170 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867173 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:29.868721 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867175 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867178 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867180 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867183 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867185 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867188 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867190 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867193 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867195 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867197 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867200 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867203 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867205 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867208 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867211 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867214 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867216 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867219 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867221 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867223 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:29.869212 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867226 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867228 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867231 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867233 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867236 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867239 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867242 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867245 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867266 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867269 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867273 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867275 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867278 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867281 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867284 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867286 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867289 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867291 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867294 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867296 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:29.869736 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867299 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867301 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867304 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867307 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867309 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867312 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867315 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867318 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867323 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867326 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867329 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867332 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867335 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867338 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867340 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867343 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867346 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867348 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867350 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:29.870209 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867353 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867355 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867358 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867363 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867373 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867376 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867378 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867381 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867383 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867386 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867389 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867391 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867394 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867396 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867399 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867401 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867404 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867406 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867409 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867412 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:29.870669 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867824 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867829 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867832 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867834 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867837 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867840 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867843 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867847 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867850 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867853 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867857 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867859 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867862 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867865 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867868 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867872 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867880 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867883 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867886 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867888 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:29.871133 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867891 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867894 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867897 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867899 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867902 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867904 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867907 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867909 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867913 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867916 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867918 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867922 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867925 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867928 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867930 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867933 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867935 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867938 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:29.871624 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867940 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867943 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867945 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867948 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867950 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867953 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867955 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867957 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867960 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867963 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867966 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867974 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867977 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867979 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867981 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867984 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867986 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867989 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867991 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867994 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:29.872043 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867996 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.867999 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868002 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868004 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868007 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868010 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868012 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868015 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868021 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868024 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868027 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868029 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868031 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868034 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868036 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868039 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868041 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868043 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868046 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868048 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:29.872516 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868050 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868055 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868058 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868062 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868071 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868075 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868077 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868080 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868082 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868085 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868088 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868090 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868093 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868095 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868098 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868100 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868104 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868106 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868109 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:29.872975 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868111 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868114 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868187 2575 flags.go:64] FLAG: --address="0.0.0.0" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868195 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868207 2575 flags.go:64] FLAG: --anonymous-auth="true" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868211 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868215 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868218 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868223 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868226 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868230 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868232 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868236 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868239 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868243 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868246 2575 flags.go:64] FLAG: --cgroup-root="" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868260 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868264 2575 flags.go:64] FLAG: --client-ca-file="" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868273 2575 flags.go:64] FLAG: --cloud-provider="external" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868276 2575 flags.go:64] FLAG: --cluster-dns="[]" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868282 2575 flags.go:64] FLAG: --cluster-domain="" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868285 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868288 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868291 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 04 09:40:29.873434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868294 2575 flags.go:64] FLAG: --container-log-max-files="5" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868298 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868300 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868303 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868306 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868310 2575 flags.go:64] FLAG: --contention-profiling="false" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868313 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868316 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868319 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868322 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868331 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868333 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868336 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868339 2575 flags.go:64] FLAG: --enable-load-reader="false" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868342 2575 flags.go:64] FLAG: --enable-server="true" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868345 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868351 2575 flags.go:64] FLAG: --event-burst="100" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868354 2575 flags.go:64] FLAG: --event-qps="50" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868357 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868360 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868363 2575 flags.go:64] FLAG: --eviction-hard="" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868367 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868370 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868373 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868376 2575 flags.go:64] FLAG: --eviction-soft="" Aug 04 09:40:29.873988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868379 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868381 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868389 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868393 2575 flags.go:64] FLAG: --experimental-mounter-path="" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868396 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868398 2575 flags.go:64] FLAG: --fail-swap-on="true" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868401 2575 flags.go:64] FLAG: --feature-gates="" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868405 2575 flags.go:64] FLAG: --file-check-frequency="20s" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868408 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868411 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868414 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868417 2575 flags.go:64] FLAG: --healthz-port="10248" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868421 2575 flags.go:64] FLAG: --help="false" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868424 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-143-10.ec2.internal" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868427 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868430 2575 flags.go:64] FLAG: --http-check-frequency="20s" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868433 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868436 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868440 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868442 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868445 2575 flags.go:64] FLAG: --image-service-endpoint="" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868448 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868450 2575 flags.go:64] FLAG: --kube-api-burst="100" Aug 04 09:40:29.874588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868453 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868456 2575 flags.go:64] FLAG: --kube-api-qps="50" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868459 2575 flags.go:64] FLAG: --kube-reserved="" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868462 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868465 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868468 2575 flags.go:64] FLAG: --kubelet-cgroups="" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868471 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868473 2575 flags.go:64] FLAG: --lock-file="" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868476 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868479 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868482 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868487 2575 flags.go:64] FLAG: --log-json-split-stream="false" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868495 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868498 2575 flags.go:64] FLAG: --log-text-split-stream="false" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868501 2575 flags.go:64] FLAG: --logging-format="text" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868504 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868507 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868510 2575 flags.go:64] FLAG: --manifest-url="" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868512 2575 flags.go:64] FLAG: --manifest-url-header="" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868517 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868519 2575 flags.go:64] FLAG: --max-open-files="1000000" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868525 2575 flags.go:64] FLAG: --max-pods="110" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868528 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868531 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868534 2575 flags.go:64] FLAG: --memory-manager-policy="None" Aug 04 09:40:29.875199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868537 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868540 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868542 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868545 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868553 2575 flags.go:64] FLAG: --node-status-max-images="50" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868555 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868558 2575 flags.go:64] FLAG: --oom-score-adj="-999" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868561 2575 flags.go:64] FLAG: --pod-cidr="" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868564 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868569 2575 flags.go:64] FLAG: --pod-manifest-path="" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868572 2575 flags.go:64] FLAG: --pod-max-pids="-1" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868575 2575 flags.go:64] FLAG: --pods-per-core="0" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868578 2575 flags.go:64] FLAG: --port="10250" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868581 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868584 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02edccaff6b210a17" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868587 2575 flags.go:64] FLAG: --qos-reserved="" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868589 2575 flags.go:64] FLAG: --read-only-port="10255" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868592 2575 flags.go:64] FLAG: --register-node="true" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868595 2575 flags.go:64] FLAG: --register-with-taints="" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868598 2575 flags.go:64] FLAG: --registry-burst="10" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868607 2575 flags.go:64] FLAG: --registry-qps="5" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868609 2575 flags.go:64] FLAG: --reserved-cpus="" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868612 2575 flags.go:64] FLAG: --reserved-memory="" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868615 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868618 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 04 09:40:29.875811 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868621 2575 flags.go:64] FLAG: --rotate-certificates="false" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868624 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868626 2575 flags.go:64] FLAG: --runonce="false" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868630 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868633 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868636 2575 flags.go:64] FLAG: --seccomp-default="false" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868639 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868642 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868645 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868648 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868650 2575 flags.go:64] FLAG: --storage-driver-password="root" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868653 2575 flags.go:64] FLAG: --storage-driver-secure="false" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868656 2575 flags.go:64] FLAG: --storage-driver-table="stats" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868658 2575 flags.go:64] FLAG: --storage-driver-user="root" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868661 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868664 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868667 2575 flags.go:64] FLAG: --system-cgroups="" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868671 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868676 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868679 2575 flags.go:64] FLAG: --tls-cert-file="" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868681 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868690 2575 flags.go:64] FLAG: --tls-min-version="" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868693 2575 flags.go:64] FLAG: --tls-private-key-file="" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868695 2575 flags.go:64] FLAG: --topology-manager-policy="none" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868698 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 04 09:40:29.876463 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868701 2575 flags.go:64] FLAG: --topology-manager-scope="container" Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868703 2575 flags.go:64] FLAG: --v="2" Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868707 2575 flags.go:64] FLAG: --version="false" Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868717 2575 flags.go:64] FLAG: --vmodule="" Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868721 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.868727 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868877 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868880 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868883 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868886 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868890 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868893 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868896 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868899 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868902 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868904 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868907 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868909 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868912 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868914 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868916 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:29.877042 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868919 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868922 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868924 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868928 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868931 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868933 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868935 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868938 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868940 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868943 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868946 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868949 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868951 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868953 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868956 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868965 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868969 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868971 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868974 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868977 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:29.877563 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868979 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868983 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868986 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868989 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868991 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868994 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.868997 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869000 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869003 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869006 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869008 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869011 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869014 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869017 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869020 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869024 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869026 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869029 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869032 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869034 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:29.878036 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869037 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869039 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869042 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869044 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869046 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869049 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869051 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869054 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869062 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869065 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869067 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869071 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869074 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869079 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869082 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869085 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869087 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869090 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:29.878554 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869093 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869095 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869099 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869102 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869105 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869107 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869110 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869112 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869115 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869118 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869121 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869123 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869126 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869128 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869131 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869133 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869136 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869138 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869141 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:29.878987 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869143 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869146 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869149 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869151 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869160 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869163 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.869165 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.869905 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.876967 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.876993 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.877014 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.877020 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877080 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877085 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:29.879450 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877091 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877095 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877099 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877101 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877104 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877107 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877110 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877113 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877115 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877118 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877121 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877123 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877126 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877129 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877131 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877134 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877137 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877139 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877142 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877144 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:29.879809 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877147 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877149 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877152 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877155 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877157 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877160 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877162 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877170 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877173 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877176 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877179 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877181 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877184 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877186 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877189 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877192 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877194 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877197 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877199 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:29.880292 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877202 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877204 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877207 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877210 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877213 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877215 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877218 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877220 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877223 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877225 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877228 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877230 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877232 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877235 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877237 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877240 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877242 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877245 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877264 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877267 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:29.880734 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877270 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877277 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877280 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877283 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877286 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877288 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877291 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877294 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877296 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877298 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877301 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877304 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877308 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877311 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877313 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877316 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877319 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877322 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877325 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:29.881200 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877327 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877330 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877332 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877335 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877337 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877340 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877342 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877345 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877348 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877350 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877352 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877355 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877357 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877361 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877364 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877370 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877374 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877377 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:29.881656 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877380 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.877386 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877506 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877510 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877513 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877516 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877519 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877522 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877524 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877527 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877529 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877532 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877534 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877537 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877539 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:29.882078 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877542 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877544 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877547 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877549 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877551 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877554 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877556 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877559 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877561 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877564 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877567 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877569 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877572 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877575 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877578 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877581 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877584 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877586 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877589 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:29.882454 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877593 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877596 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877599 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877601 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877604 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877607 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877610 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877612 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877614 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877617 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877619 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877621 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877624 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877627 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877629 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877632 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877634 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877636 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877639 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877641 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:29.882902 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877644 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877646 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877649 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877651 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877654 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877657 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877659 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877662 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877664 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877667 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877670 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877672 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877675 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877677 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877680 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877682 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877686 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877689 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877692 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:29.883501 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877694 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877697 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877700 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877702 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877705 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877708 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877711 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877713 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877716 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877718 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877720 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877723 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877725 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877728 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877730 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877733 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877737 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877740 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877743 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877746 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:29.883953 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877748 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877751 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877753 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877756 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877759 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877761 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877764 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:29.877766 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.877771 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.877905 2575 server.go:962] "Client rotation is on, will bootstrap in background" Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.881377 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.882770 2575 server.go:1019] "Starting client certificate rotation" Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.882874 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 09:40:29.884435 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.883605 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 09:40:29.885392 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.885377 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 09:40:29.890081 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.890048 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 09:40:29.900635 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.900607 2575 log.go:25] "Validated CRI v1 runtime API" Aug 04 09:40:29.904084 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.904067 2575 log.go:25] "Validated CRI v1 image API" Aug 04 09:40:29.904882 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.904867 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 04 09:40:29.907278 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.907243 2575 fs.go:135] Filesystem UUIDs: map[375f0400-0b70-4f41-a9a5-bd1495db53a0:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 83dc89f7-03b9-4e28-bbfd-128f63f27bfa:/dev/nvme0n1p4] Aug 04 09:40:29.907333 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.907278 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 04 09:40:29.912923 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.912782 2575 manager.go:216] Machine: {Timestamp:2026-08-04 09:40:29.911460986 +0000 UTC m=+0.300230699 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3212746 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f2f59b6d2260941424e352e46a0a6 SystemUUID:ec2f2f59-b6d2-2609-4142-4e352e46a0a6 BootID:30cf3d12-c8ac-4d95-8524-4a734ec95d83 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e4:33:58:fb:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e4:33:58:fb:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:d1:f3:cd:37:9b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 04 09:40:29.912923 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.912906 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 04 09:40:29.913059 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.913038 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 04 09:40:29.916819 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.916782 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 04 09:40:29.916975 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.916821 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-10.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 04 09:40:29.917361 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.917345 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 09:40:29.918185 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.918174 2575 topology_manager.go:138] "Creating topology manager with none policy" Aug 04 09:40:29.918217 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.918189 2575 container_manager_linux.go:309] "Creating device plugin manager" Aug 04 09:40:29.918217 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.918197 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 09:40:29.918927 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.918916 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 09:40:29.920060 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.920045 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 04 09:40:29.925011 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.924995 2575 state_mem.go:36] "Initialized new in-memory state store" Aug 04 09:40:29.925155 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.925145 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 04 09:40:29.925195 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.925176 2575 kubelet.go:486] "Attempting to sync node with API server" Aug 04 09:40:29.925195 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.925193 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 04 09:40:29.925269 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.925209 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 04 09:40:29.925269 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.925219 2575 kubelet.go:398] "Adding apiserver pod source" Aug 04 09:40:29.925269 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.925229 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 04 09:40:29.925704 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.925691 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 09:40:29.925737 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.925715 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 09:40:29.928065 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.928048 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 04 09:40:29.929994 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.929980 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 04 09:40:29.930052 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.930009 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 04 09:40:29.931392 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931376 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 04 09:40:29.931392 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931393 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 04 09:40:29.931515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931399 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 04 09:40:29.931515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931405 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 04 09:40:29.931515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931414 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 04 09:40:29.931515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931420 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 04 09:40:29.931515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931429 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 04 09:40:29.931515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931434 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 04 09:40:29.931515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931445 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 04 09:40:29.931515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931450 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 04 09:40:29.931515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931463 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 04 09:40:29.931515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.931468 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 04 09:40:29.933028 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.933016 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 04 09:40:29.933028 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.933028 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 04 09:40:29.936192 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.936166 2575 server.go:1268] "Started kubelet" Aug 04 09:40:29.936306 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.936271 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 04 09:40:29.936344 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.936303 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 04 09:40:29.936385 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.936363 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 04 09:40:29.940955 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.940927 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 04 09:40:29.941634 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:29.941602 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 04 09:40:29.941867 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.941849 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 04 09:40:29.942043 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:29.942020 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 04 09:40:29.942385 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:29.942364 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 04 09:40:29.942460 ip-10-0-143-10 systemd[1]: Started Kubernetes Kubelet. Aug 04 09:40:29.943560 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.943545 2575 server.go:310] "Adding debug handlers to kubelet server" Aug 04 09:40:29.947167 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:29.946073 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-10.ec2.internal.18c890c5714b898a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-10.ec2.internal,UID:ip-10-0-143-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-10.ec2.internal,},FirstTimestamp:2026-08-04 09:40:29.936134538 +0000 UTC m=+0.324904251,LastTimestamp:2026-08-04 09:40:29.936134538 +0000 UTC m=+0.324904251,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-10.ec2.internal,}" Aug 04 09:40:29.956439 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.956408 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 04 09:40:29.956439 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.956411 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 04 09:40:29.956644 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:29.956623 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-10.ec2.internal\" not found" Aug 04 09:40:29.957093 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957070 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 04 09:40:29.957093 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957076 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 04 09:40:29.957093 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957100 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 04 09:40:29.957320 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957210 2575 reconstruct.go:97] "Volume reconstruction finished" Aug 04 09:40:29.957320 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957219 2575 reconciler.go:29] "Reconciler: start to sync state" Aug 04 09:40:29.957547 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957531 2575 factory.go:153] Registering CRI-O factory Aug 04 09:40:29.957616 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957550 2575 factory.go:223] Registration of the crio container factory successfully Aug 04 09:40:29.957616 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957610 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 04 09:40:29.957693 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957620 2575 factory.go:55] Registering systemd factory Aug 04 09:40:29.957693 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957628 2575 factory.go:223] Registration of the systemd container factory successfully Aug 04 09:40:29.957693 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957647 2575 factory.go:103] Registering Raw factory Aug 04 09:40:29.957693 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.957661 2575 manager.go:1196] Started watching for new ooms in manager Aug 04 09:40:29.958066 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.958052 2575 manager.go:349] Starting recovery of all containers Aug 04 09:40:29.959933 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:29.959899 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 04 09:40:29.960027 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:29.959954 2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Aug 04 09:40:29.967922 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.967899 2575 manager.go:354] Recovery completed Aug 04 09:40:29.973483 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.973468 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:29.975690 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.975674 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:29.975754 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.975705 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:29.975754 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.975719 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:29.976343 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.976326 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 04 09:40:29.976343 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.976340 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 04 09:40:29.976517 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.976348 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:29.976517 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.976374 2575 state_mem.go:36] "Initialized new in-memory state store" Aug 04 09:40:29.977110 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.977093 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:29.977177 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.977121 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:29.977177 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.977130 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:29.978678 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.978667 2575 policy_none.go:49] "None policy: Start" Aug 04 09:40:29.978720 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.978682 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 04 09:40:29.978720 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.978692 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 04 09:40:29.980759 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:29.980748 2575 policy_none.go:47] "Start" Aug 04 09:40:29.989898 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:29.989815 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-10.ec2.internal.18c890c573a71bf4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-10.ec2.internal,UID:ip-10-0-143-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-10.ec2.internal,},FirstTimestamp:2026-08-04 09:40:29.975690228 +0000 UTC m=+0.364459941,LastTimestamp:2026-08-04 09:40:29.975690228 +0000 UTC m=+0.364459941,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-10.ec2.internal,}" Aug 04 09:40:29.994575 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:29.994473 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-10.ec2.internal.18c890c573a7735e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-10.ec2.internal,UID:ip-10-0-143-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-10.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-10.ec2.internal,},FirstTimestamp:2026-08-04 09:40:29.975712606 +0000 UTC m=+0.364482324,LastTimestamp:2026-08-04 09:40:29.975712606 +0000 UTC m=+0.364482324,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-10.ec2.internal,}" Aug 04 09:40:30.003677 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.003369 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-10.ec2.internal.18c890c573a79c06 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-10.ec2.internal,UID:ip-10-0-143-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-10.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-10.ec2.internal,},FirstTimestamp:2026-08-04 09:40:29.975723014 +0000 UTC m=+0.364492727,LastTimestamp:2026-08-04 09:40:29.975723014 +0000 UTC m=+0.364492727,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-10.ec2.internal,}" Aug 04 09:40:30.009216 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.009138 2575 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-10.ec2.internal.18c890c573a71bf4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-10.ec2.internal.18c890c573a71bf4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-10.ec2.internal,UID:ip-10-0-143-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-10.ec2.internal,},FirstTimestamp:2026-08-04 09:40:29.975690228 +0000 UTC m=+0.364459941,LastTimestamp:2026-08-04 09:40:29.977108309 +0000 UTC m=+0.365878021,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-10.ec2.internal,}" Aug 04 09:40:30.016481 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.016406 2575 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-10.ec2.internal.18c890c573a7735e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-10.ec2.internal.18c890c573a7735e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-10.ec2.internal,UID:ip-10-0-143-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-10.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-10.ec2.internal,},FirstTimestamp:2026-08-04 09:40:29.975712606 +0000 UTC m=+0.364482324,LastTimestamp:2026-08-04 09:40:29.97712537 +0000 UTC m=+0.365895082,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-10.ec2.internal,}" Aug 04 09:40:30.018984 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.018968 2575 manager.go:337] "Starting Device Plugin manager" Aug 04 09:40:30.019039 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.019020 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 04 09:40:30.019039 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.019034 2575 server.go:87] "Starting device plugin registration server" Aug 04 09:40:30.019335 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.019324 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 04 09:40:30.019387 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.019336 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 04 09:40:30.019521 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.019505 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 04 09:40:30.019592 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.019586 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 04 09:40:30.019655 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.019595 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 04 09:40:30.020162 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.020139 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 04 09:40:30.020270 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.020194 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-10.ec2.internal\" not found" Aug 04 09:40:30.024179 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.024113 2575 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-10.ec2.internal.18c890c573a79c06\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-10.ec2.internal.18c890c573a79c06 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-10.ec2.internal,UID:ip-10-0-143-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-10.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-10.ec2.internal,},FirstTimestamp:2026-08-04 09:40:29.975723014 +0000 UTC m=+0.364492727,LastTimestamp:2026-08-04 09:40:29.97713412 +0000 UTC m=+0.365903837,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-10.ec2.internal,}" Aug 04 09:40:30.027914 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.027850 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-10.ec2.internal.18c890c57662c467 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-10.ec2.internal,UID:ip-10-0-143-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-143-10.ec2.internal,},FirstTimestamp:2026-08-04 09:40:30.021543015 +0000 UTC m=+0.410312722,LastTimestamp:2026-08-04 09:40:30.021543015 +0000 UTC m=+0.410312722,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-10.ec2.internal,}" Aug 04 09:40:30.086586 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.086548 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d7z4p" Aug 04 09:40:30.097262 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.097223 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d7z4p" Aug 04 09:40:30.120173 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.120076 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:30.121014 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.120998 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:30.121084 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.121026 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:30.121084 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.121035 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:30.121084 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.121062 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.123439 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.123349 2575 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-10.ec2.internal.18c890c573a71bf4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-10.ec2.internal.18c890c573a71bf4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-10.ec2.internal,UID:ip-10-0-143-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-10.ec2.internal,},FirstTimestamp:2026-08-04 09:40:29.975690228 +0000 UTC m=+0.364459941,LastTimestamp:2026-08-04 09:40:30.121013108 +0000 UTC m=+0.509782821,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-10.ec2.internal,}" Aug 04 09:40:30.132979 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.132945 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 04 09:40:30.134289 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.134244 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 04 09:40:30.134372 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.134296 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 04 09:40:30.134372 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.134322 2575 kubelet.go:2448] "Starting kubelet main sync loop" Aug 04 09:40:30.134372 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.134358 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 04 09:40:30.142554 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.142522 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:40:30.151078 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.151057 2575 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-10.ec2.internal\" not found" node="ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.174201 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.174174 2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-10.ec2.internal\" not found" node="ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.235416 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.235387 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-10.ec2.internal"] Aug 04 09:40:30.235519 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.235505 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:30.236431 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.236414 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:30.236507 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.236444 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:30.236507 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.236455 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:30.238812 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.238799 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:30.238923 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.238908 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.238959 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.238945 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:30.239652 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.239636 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:30.239652 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.239647 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:30.239652 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.239656 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:30.239812 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.239668 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:30.239812 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.239669 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:30.239812 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.239700 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:30.241883 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.241868 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.241971 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.241894 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:30.242643 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.242630 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:30.242720 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.242656 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:30.242720 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.242669 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:30.258708 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.258675 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3abd502c334bfa63fadf295cd4cdea9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal\" (UID: \"3abd502c334bfa63fadf295cd4cdea9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.258708 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.258710 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3abd502c334bfa63fadf295cd4cdea9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal\" (UID: \"3abd502c334bfa63fadf295cd4cdea9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.258899 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.258738 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/499f0d85eada0e49ab159e031fad0580-config\") pod \"kube-apiserver-proxy-ip-10-0-143-10.ec2.internal\" (UID: \"499f0d85eada0e49ab159e031fad0580\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.270382 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.270357 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-10.ec2.internal\" not found" node="ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.274810 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.274792 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-10.ec2.internal\" not found" node="ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.352045 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.352019 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:30.353137 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.353120 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:30.353226 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.353148 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:30.353226 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.353159 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:30.353226 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.353187 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.358870 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.358848 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3abd502c334bfa63fadf295cd4cdea9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal\" (UID: \"3abd502c334bfa63fadf295cd4cdea9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.358926 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.358878 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3abd502c334bfa63fadf295cd4cdea9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal\" (UID: \"3abd502c334bfa63fadf295cd4cdea9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.358926 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.358897 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/499f0d85eada0e49ab159e031fad0580-config\") pod \"kube-apiserver-proxy-ip-10-0-143-10.ec2.internal\" (UID: \"499f0d85eada0e49ab159e031fad0580\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.358994 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.358938 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/499f0d85eada0e49ab159e031fad0580-config\") pod \"kube-apiserver-proxy-ip-10-0-143-10.ec2.internal\" (UID: \"499f0d85eada0e49ab159e031fad0580\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.358994 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.358950 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3abd502c334bfa63fadf295cd4cdea9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal\" (UID: \"3abd502c334bfa63fadf295cd4cdea9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.358994 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.358963 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3abd502c334bfa63fadf295cd4cdea9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal\" (UID: \"3abd502c334bfa63fadf295cd4cdea9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.367177 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:30.367152 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-10.ec2.internal" Aug 04 09:40:30.367236 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.367180 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-10.ec2.internal\": node \"ip-10-0-143-10.ec2.internal\" not found" Aug 04 09:40:30.387899 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.387829 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-10.ec2.internal\" not found" Aug 04 09:40:30.488407 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:30.488373 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-10.ec2.internal\" not found" Aug 04 09:40:31.770421 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:31.770387 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" Aug 04 09:40:31.776338 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:31.776313 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-10.ec2.internal" Aug 04 09:40:31.786338 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:31.786316 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-10.ec2.internal\" not found" Aug 04 09:40:31.886972 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:31.886870 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-10.ec2.internal\" not found" Aug 04 09:40:31.987395 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:31.987341 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-10.ec2.internal\" not found" Aug 04 09:40:32.080087 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.080048 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 04 09:40:32.080535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.080197 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 09:40:32.088367 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:32.088332 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-10.ec2.internal\" not found" Aug 04 09:40:32.127765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.127740 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:40:32.154213 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.154149 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 04 09:40:32.154312 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.154158 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" Aug 04 09:40:32.182250 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.182225 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 09:40:32.184208 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.184193 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-10.ec2.internal" Aug 04 09:40:32.194127 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.194101 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 09:40:32.201019 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.201000 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 09:40:32.268150 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.268117 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fth44" Aug 04 09:40:32.275686 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.275663 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fth44" Aug 04 09:40:32.300679 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.300641 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 09:35:31 +0000 UTC" deadline="2028-02-20 02:03:21.087439732 +0000 UTC" Aug 04 09:40:32.300679 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.300671 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13552h22m48.786771155s" Aug 04 09:40:32.320049 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.320028 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:40:32.430087 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:32.430050 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod499f0d85eada0e49ab159e031fad0580.slice/crio-00cc5763499b28246527c8ac7c68e2eb009fb8343ff460f8c8f01697b5cca2a3 WatchSource:0}: Error finding container 00cc5763499b28246527c8ac7c68e2eb009fb8343ff460f8c8f01697b5cca2a3: Status 404 returned error can't find the container with id 00cc5763499b28246527c8ac7c68e2eb009fb8343ff460f8c8f01697b5cca2a3 Aug 04 09:40:32.436780 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.436762 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 09:40:32.493488 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:32.493458 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:40:33.097139 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:33.097102 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3abd502c334bfa63fadf295cd4cdea9c.slice/crio-b9395516e7232c7349afec35976dc163abaa84f84f0ea46419a18b399f77f7ea WatchSource:0}: Error finding container b9395516e7232c7349afec35976dc163abaa84f84f0ea46419a18b399f77f7ea: Status 404 returned error can't find the container with id b9395516e7232c7349afec35976dc163abaa84f84f0ea46419a18b399f77f7ea Aug 04 09:40:33.124370 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.124324 2575 apiserver.go:52] "Watching apiserver" Aug 04 09:40:33.131692 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.131662 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 04 09:40:33.132132 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.132103 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-c6bvn","openshift-network-operator/iptables-alerter-t6qlp","kube-system/global-pull-secret-syncer-g6s9t","kube-system/konnectivity-agent-lbmxb","openshift-image-registry/node-ca-slfll","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal","openshift-multus/multus-vczmz","openshift-ovn-kubernetes/ovnkube-node-rb78b","kube-system/kube-apiserver-proxy-ip-10-0-143-10.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942","openshift-cluster-node-tuning-operator/tuned-24dll","openshift-dns/node-resolver-vkqkd","openshift-multus/multus-additional-cni-plugins-qmwhl","openshift-multus/network-metrics-daemon-pcs94"] Aug 04 09:40:33.135023 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.134972 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.137158 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.137129 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.139445 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.139309 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:33.139445 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.139395 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:33.141890 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.141592 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:40:33.142242 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.142190 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 04 09:40:33.142242 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.142209 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 04 09:40:33.142242 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.142224 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 04 09:40:33.142447 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.142269 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 04 09:40:33.142500 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.142473 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dz4lf\"" Aug 04 09:40:33.144113 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.144094 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-slfll" Aug 04 09:40:33.146366 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.146347 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:33.146487 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.146453 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:33.148855 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.148834 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.151399 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.151295 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:33.151399 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.151357 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:33.153616 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.153597 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t6qlp" Aug 04 09:40:33.156186 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.156167 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.159748 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.158919 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.161325 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.161303 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vkqkd" Aug 04 09:40:33.163786 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.163466 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 04 09:40:33.163786 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.163527 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 04 09:40:33.163786 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.163704 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 04 09:40:33.164714 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.164339 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 04 09:40:33.164714 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.164379 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 04 09:40:33.164714 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.164483 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 04 09:40:33.164714 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.164583 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:40:33.164714 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.164602 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-npfrv\"" Aug 04 09:40:33.165017 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.164782 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:40:33.165017 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.164911 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 04 09:40:33.165111 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165058 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-run-ovn-kubernetes\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.165162 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165138 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-slash\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.165214 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165167 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/75063389-386c-4849-9baf-c001a3c3f9d2-agent-certs\") pod \"konnectivity-agent-lbmxb\" (UID: \"75063389-386c-4849-9baf-c001a3c3f9d2\") " pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:40:33.165280 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165259 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-sys-fs\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.165331 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165295 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/75063389-386c-4849-9baf-c001a3c3f9d2-konnectivity-ca\") pod \"konnectivity-agent-lbmxb\" (UID: \"75063389-386c-4849-9baf-c001a3c3f9d2\") " pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:40:33.165331 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165323 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 04 09:40:33.165419 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvpqt\" (UniqueName: \"kubernetes.io/projected/f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222-kube-api-access-tvpqt\") pod \"iptables-alerter-t6qlp\" (UID: \"f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222\") " pod="openshift-network-operator/iptables-alerter-t6qlp" Aug 04 09:40:33.165419 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165344 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 04 09:40:33.165419 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165361 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b16b5696-7aaa-4c86-928c-09e2f6d8f8b4-serviceca\") pod \"node-ca-slfll\" (UID: \"b16b5696-7aaa-4c86-928c-09e2f6d8f8b4\") " pod="openshift-image-registry/node-ca-slfll" Aug 04 09:40:33.165556 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165419 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:33.165556 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165477 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-run-netns\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.165556 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165486 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 04 09:40:33.165556 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165510 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 04 09:40:33.166140 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.165913 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-w25xg\"" Aug 04 09:40:33.166227 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.166198 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vp76h\"" Aug 04 09:40:33.166408 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.166394 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vm472\"" Aug 04 09:40:33.166752 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.166647 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-86j4v\"" Aug 04 09:40:33.166919 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.166852 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 04 09:40:33.167084 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167068 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-z6xsp\"" Aug 04 09:40:33.167429 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167411 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-var-lib-cni-multus\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.167498 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167477 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:33.167550 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167510 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-var-lib-openvswitch\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.167606 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167549 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-ovnkube-script-lib\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.167606 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167574 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-etc-kubernetes\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.167699 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167648 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6309670e-2dff-4216-b018-1ed218e5da34-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.167765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167706 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-env-overrides\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.167765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167751 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg6gv\" (UniqueName: \"kubernetes.io/projected/250d4003-cf01-4a28-9b5e-128d047c2260-kube-api-access-bg6gv\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.167932 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167800 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-os-release\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.167932 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167926 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-run-ovn\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.168025 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.167960 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tvpw\" (UniqueName: \"kubernetes.io/projected/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-kube-api-access-6tvpw\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.168025 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.168001 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-hostroot\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.168116 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.168093 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/250d4003-cf01-4a28-9b5e-128d047c2260-multus-daemon-config\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.168168 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.168131 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6309670e-2dff-4216-b018-1ed218e5da34-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.168315 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.168299 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6309670e-2dff-4216-b018-1ed218e5da34-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.168370 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.168328 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-etc-selinux\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.168414 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.168390 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27m4z\" (UniqueName: \"kubernetes.io/projected/fed15a75-90cc-42bc-831e-e343ac873cf6-kube-api-access-27m4z\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.168870 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.168828 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-node-log\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.169000 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.168984 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2004042-19e9-46e9-8379-ad03ca29d486-kubelet-config\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:33.169043 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169026 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222-iptables-alerter-script\") pod \"iptables-alerter-t6qlp\" (UID: \"f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222\") " pod="openshift-network-operator/iptables-alerter-t6qlp" Aug 04 09:40:33.169103 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169088 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-cnibin\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.169152 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169127 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppddl\" (UniqueName: \"kubernetes.io/projected/b16b5696-7aaa-4c86-928c-09e2f6d8f8b4-kube-api-access-ppddl\") pod \"node-ca-slfll\" (UID: \"b16b5696-7aaa-4c86-928c-09e2f6d8f8b4\") " pod="openshift-image-registry/node-ca-slfll" Aug 04 09:40:33.169326 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169312 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-systemd-units\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.169382 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169363 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-cni-bin\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.169418 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169407 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-cni-netd\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.169457 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169441 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-var-lib-kubelet\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.169590 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169576 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-run-multus-certs\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.169625 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169604 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 04 09:40:33.169753 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2004042-19e9-46e9-8379-ad03ca29d486-dbus\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:33.169821 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-system-cni-dir\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.169872 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169830 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-kubelet\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.169917 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169868 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-ovnkube-config\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.169917 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169894 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 04 09:40:33.169917 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169911 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/250d4003-cf01-4a28-9b5e-128d047c2260-cni-binary-copy\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.170062 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169941 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-var-lib-cni-bin\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.170062 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169958 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 04 09:40:33.170062 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169994 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 04 09:40:33.170062 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.170048 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 04 09:40:33.170194 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.169892 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 04 09:40:33.170313 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.170281 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-multus-conf-dir\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.170652 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.170318 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85nnq\" (UniqueName: \"kubernetes.io/projected/6309670e-2dff-4216-b018-1ed218e5da34-kube-api-access-85nnq\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.170652 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.170350 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.170652 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.170456 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-ovn-node-metrics-cert\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.170652 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.170515 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-multus-cni-dir\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.170889 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.170689 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-cnibin\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.170889 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.170764 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-run-systemd\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.170889 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.170797 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-run-openvswitch\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.171361 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.171026 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 04 09:40:33.171361 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.171056 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-socket-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.171361 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.171146 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hwpqk\"" Aug 04 09:40:33.171361 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.171122 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twxtp\" (UniqueName: \"kubernetes.io/projected/f192c223-2f86-4ee1-836a-60af7de6824a-kube-api-access-twxtp\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:33.171361 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.171226 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-log-socket\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.171361 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.171327 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.171761 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.171364 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-system-cni-dir\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.171761 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.171419 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-multus-socket-dir-parent\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.171761 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.171454 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-run-netns\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.172078 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.171986 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b16b5696-7aaa-4c86-928c-09e2f6d8f8b4-host\") pod \"node-ca-slfll\" (UID: \"b16b5696-7aaa-4c86-928c-09e2f6d8f8b4\") " pod="openshift-image-registry/node-ca-slfll" Aug 04 09:40:33.172078 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.172047 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-os-release\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.172233 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.172131 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tj67\" (UniqueName: \"kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67\") pod \"network-check-target-c6bvn\" (UID: \"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7\") " pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:33.172233 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.172169 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.172233 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.172199 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222-host-slash\") pod \"iptables-alerter-t6qlp\" (UID: \"f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222\") " pod="openshift-network-operator/iptables-alerter-t6qlp" Aug 04 09:40:33.172233 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.172229 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-registration-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.172430 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.172269 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-device-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.172430 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.172297 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-run-k8s-cni-cncf-io\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.172430 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.172313 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 04 09:40:33.172430 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.172319 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-etc-openvswitch\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.172616 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.172525 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zr5l8\"" Aug 04 09:40:33.172616 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.172557 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 04 09:40:33.255669 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.255633 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 04 09:40:33.272997 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.272948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/75063389-386c-4849-9baf-c001a3c3f9d2-konnectivity-ca\") pod \"konnectivity-agent-lbmxb\" (UID: \"75063389-386c-4849-9baf-c001a3c3f9d2\") " pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:40:33.273159 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273019 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvpqt\" (UniqueName: \"kubernetes.io/projected/f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222-kube-api-access-tvpqt\") pod \"iptables-alerter-t6qlp\" (UID: \"f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222\") " pod="openshift-network-operator/iptables-alerter-t6qlp" Aug 04 09:40:33.273159 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273050 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-sysconfig\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.273159 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273072 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-modprobe-d\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.273159 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273103 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-var-lib-kubelet\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.273371 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273172 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b16b5696-7aaa-4c86-928c-09e2f6d8f8b4-serviceca\") pod \"node-ca-slfll\" (UID: \"b16b5696-7aaa-4c86-928c-09e2f6d8f8b4\") " pod="openshift-image-registry/node-ca-slfll" Aug 04 09:40:33.273371 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273221 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:33.273371 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273244 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-run-netns\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.273371 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273264 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-var-lib-cni-multus\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.273371 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273286 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:33.273371 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273311 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-run\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.273371 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273334 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-var-lib-openvswitch\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.273371 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273355 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-ovnkube-script-lib\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.273371 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.273364 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273379 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-etc-kubernetes\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273402 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6309670e-2dff-4216-b018-1ed218e5da34-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.273425 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273447 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-sysctl-conf\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.273483 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret podName:a2004042-19e9-46e9-8379-ad03ca29d486 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:33.773460857 +0000 UTC m=+2.964753349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret") pod "global-pull-secret-syncer-g6s9t" (UID: "a2004042-19e9-46e9-8379-ad03ca29d486") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273504 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-env-overrides\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273512 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-run-netns\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273514 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-var-lib-cni-multus\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273532 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg6gv\" (UniqueName: \"kubernetes.io/projected/250d4003-cf01-4a28-9b5e-128d047c2260-kube-api-access-bg6gv\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273554 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-var-lib-openvswitch\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273559 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-os-release\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273719 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/75063389-386c-4849-9baf-c001a3c3f9d2-konnectivity-ca\") pod \"konnectivity-agent-lbmxb\" (UID: \"75063389-386c-4849-9baf-c001a3c3f9d2\") " pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:40:33.273799 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273753 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b16b5696-7aaa-4c86-928c-09e2f6d8f8b4-serviceca\") pod \"node-ca-slfll\" (UID: \"b16b5696-7aaa-4c86-928c-09e2f6d8f8b4\") " pod="openshift-image-registry/node-ca-slfll" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273856 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-etc-kubernetes\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.273865 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs podName:f192c223-2f86-4ee1-836a-60af7de6824a nodeName:}" failed. No retries permitted until 2026-08-04 09:40:33.773840065 +0000 UTC m=+2.965132554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs") pod "network-metrics-daemon-pcs94" (UID: "f192c223-2f86-4ee1-836a-60af7de6824a") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273957 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-os-release\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.273976 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-run-ovn\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274010 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tvpw\" (UniqueName: \"kubernetes.io/projected/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-kube-api-access-6tvpw\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-hostroot\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274071 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-run-ovn\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274077 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/250d4003-cf01-4a28-9b5e-128d047c2260-multus-daemon-config\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274116 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6309670e-2dff-4216-b018-1ed218e5da34-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274165 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6309670e-2dff-4216-b018-1ed218e5da34-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274173 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-ovnkube-script-lib\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274198 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8612b3d8-d52c-4b28-96b1-c9202698e69d-tmp-dir\") pod \"node-resolver-vkqkd\" (UID: \"8612b3d8-d52c-4b28-96b1-c9202698e69d\") " pod="openshift-dns/node-resolver-vkqkd" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274202 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-env-overrides\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274235 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-hostroot\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274228 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-etc-selinux\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274278 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27m4z\" (UniqueName: \"kubernetes.io/projected/fed15a75-90cc-42bc-831e-e343ac873cf6-kube-api-access-27m4z\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.274386 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274304 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-node-log\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274334 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-etc-selinux\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274341 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2004042-19e9-46e9-8379-ad03ca29d486-kubelet-config\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274408 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222-iptables-alerter-script\") pod \"iptables-alerter-t6qlp\" (UID: \"f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222\") " pod="openshift-network-operator/iptables-alerter-t6qlp" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-cnibin\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274361 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6309670e-2dff-4216-b018-1ed218e5da34-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274455 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2004042-19e9-46e9-8379-ad03ca29d486-kubelet-config\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274401 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-node-log\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274511 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-tmp\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274516 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-cnibin\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274538 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppddl\" (UniqueName: \"kubernetes.io/projected/b16b5696-7aaa-4c86-928c-09e2f6d8f8b4-kube-api-access-ppddl\") pod \"node-ca-slfll\" (UID: \"b16b5696-7aaa-4c86-928c-09e2f6d8f8b4\") " pod="openshift-image-registry/node-ca-slfll" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274561 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-systemd-units\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274585 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-cni-bin\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-cni-netd\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274630 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-var-lib-kubelet\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274675 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-run-multus-certs\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274679 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-systemd-units\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.275138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274699 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2004042-19e9-46e9-8379-ad03ca29d486-dbus\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274743 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-system-cni-dir\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274748 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-var-lib-kubelet\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274774 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9svj\" (UniqueName: \"kubernetes.io/projected/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-kube-api-access-d9svj\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274799 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-cni-bin\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274810 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-cni-netd\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274801 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8612b3d8-d52c-4b28-96b1-c9202698e69d-hosts-file\") pod \"node-resolver-vkqkd\" (UID: \"8612b3d8-d52c-4b28-96b1-c9202698e69d\") " pod="openshift-dns/node-resolver-vkqkd" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274723 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/250d4003-cf01-4a28-9b5e-128d047c2260-multus-daemon-config\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274675 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6309670e-2dff-4216-b018-1ed218e5da34-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274848 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6309670e-2dff-4216-b018-1ed218e5da34-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274853 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-system-cni-dir\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274849 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-run-multus-certs\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274887 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-kubelet\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274924 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-kubelet\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274926 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-ovnkube-config\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274965 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/250d4003-cf01-4a28-9b5e-128d047c2260-cni-binary-copy\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274966 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222-iptables-alerter-script\") pod \"iptables-alerter-t6qlp\" (UID: \"f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222\") " pod="openshift-network-operator/iptables-alerter-t6qlp" Aug 04 09:40:33.275894 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.274991 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-var-lib-cni-bin\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275039 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-var-lib-cni-bin\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275044 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2004042-19e9-46e9-8379-ad03ca29d486-dbus\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275053 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-multus-conf-dir\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275079 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85nnq\" (UniqueName: \"kubernetes.io/projected/6309670e-2dff-4216-b018-1ed218e5da34-kube-api-access-85nnq\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275117 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-sysctl-d\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275160 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275183 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-multus-conf-dir\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275188 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-ovn-node-metrics-cert\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275230 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-multus-cni-dir\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275232 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275261 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-cnibin\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275301 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-multus-cni-dir\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275318 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spkkj\" (UniqueName: \"kubernetes.io/projected/8612b3d8-d52c-4b28-96b1-c9202698e69d-kube-api-access-spkkj\") pod \"node-resolver-vkqkd\" (UID: \"8612b3d8-d52c-4b28-96b1-c9202698e69d\") " pod="openshift-dns/node-resolver-vkqkd" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275344 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-run-systemd\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275368 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-cnibin\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275344 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-ovnkube-config\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.276676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275368 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-run-openvswitch\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275404 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-run-systemd\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275419 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-socket-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275455 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-run-openvswitch\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275455 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/250d4003-cf01-4a28-9b5e-128d047c2260-cni-binary-copy\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275468 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twxtp\" (UniqueName: \"kubernetes.io/projected/f192c223-2f86-4ee1-836a-60af7de6824a-kube-api-access-twxtp\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275515 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-systemd\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275546 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-socket-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275697 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-log-socket\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275753 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275767 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275824 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-system-cni-dir\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275869 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-log-socket\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275883 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275777 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-system-cni-dir\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275915 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-multus-socket-dir-parent\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275931 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-run-netns\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275955 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-tuned\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.277491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275975 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b16b5696-7aaa-4c86-928c-09e2f6d8f8b4-host\") pod \"node-ca-slfll\" (UID: \"b16b5696-7aaa-4c86-928c-09e2f6d8f8b4\") " pod="openshift-image-registry/node-ca-slfll" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.275993 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-run-netns\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276000 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-os-release\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276009 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-multus-socket-dir-parent\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276048 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tj67\" (UniqueName: \"kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67\") pod \"network-check-target-c6bvn\" (UID: \"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7\") " pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276074 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-os-release\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276100 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b16b5696-7aaa-4c86-928c-09e2f6d8f8b4-host\") pod \"node-ca-slfll\" (UID: \"b16b5696-7aaa-4c86-928c-09e2f6d8f8b4\") " pod="openshift-image-registry/node-ca-slfll" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276142 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276170 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-kubernetes\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276196 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222-host-slash\") pod \"iptables-alerter-t6qlp\" (UID: \"f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222\") " pod="openshift-network-operator/iptables-alerter-t6qlp" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276220 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-registration-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276244 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-device-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276261 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222-host-slash\") pod \"iptables-alerter-t6qlp\" (UID: \"f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222\") " pod="openshift-network-operator/iptables-alerter-t6qlp" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276269 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-run-k8s-cni-cncf-io\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276277 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6309670e-2dff-4216-b018-1ed218e5da34-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276315 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/250d4003-cf01-4a28-9b5e-128d047c2260-host-run-k8s-cni-cncf-io\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276332 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-registration-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.278383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276323 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-device-dir\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276362 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-etc-openvswitch\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276387 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-run-ovn-kubernetes\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276410 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-slash\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276431 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-etc-openvswitch\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276432 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/75063389-386c-4849-9baf-c001a3c3f9d2-agent-certs\") pod \"konnectivity-agent-lbmxb\" (UID: \"75063389-386c-4849-9baf-c001a3c3f9d2\") " pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276466 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-sys-fs\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276487 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-sys\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276497 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-run-ovn-kubernetes\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276534 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-lib-modules\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276554 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-host\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276566 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fed15a75-90cc-42bc-831e-e343ac873cf6-sys-fs\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.276646 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-host-slash\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.278637 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 09:35:32 +0000 UTC" deadline="2028-04-04 01:30:40.035048688 +0000 UTC" Aug 04 09:40:33.279151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.278678 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14607h50m6.756374304s" Aug 04 09:40:33.279955 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.279931 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/75063389-386c-4849-9baf-c001a3c3f9d2-agent-certs\") pod \"konnectivity-agent-lbmxb\" (UID: \"75063389-386c-4849-9baf-c001a3c3f9d2\") " pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:40:33.280159 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.280143 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-ovn-node-metrics-cert\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.283025 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.283000 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvpqt\" (UniqueName: \"kubernetes.io/projected/f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222-kube-api-access-tvpqt\") pod \"iptables-alerter-t6qlp\" (UID: \"f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222\") " pod="openshift-network-operator/iptables-alerter-t6qlp" Aug 04 09:40:33.283550 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.283530 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg6gv\" (UniqueName: \"kubernetes.io/projected/250d4003-cf01-4a28-9b5e-128d047c2260-kube-api-access-bg6gv\") pod \"multus-vczmz\" (UID: \"250d4003-cf01-4a28-9b5e-128d047c2260\") " pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.286647 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.286551 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:33.286647 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.286575 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:33.286647 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.286589 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6tj67 for pod openshift-network-diagnostics/network-check-target-c6bvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:33.287072 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.286832 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67 podName:9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:33.786711929 +0000 UTC m=+2.978004426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6tj67" (UniqueName: "kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67") pod "network-check-target-c6bvn" (UID: "9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:33.287142 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.287114 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27m4z\" (UniqueName: \"kubernetes.io/projected/fed15a75-90cc-42bc-831e-e343ac873cf6-kube-api-access-27m4z\") pod \"aws-ebs-csi-driver-node-s5942\" (UID: \"fed15a75-90cc-42bc-831e-e343ac873cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.287701 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.287679 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppddl\" (UniqueName: \"kubernetes.io/projected/b16b5696-7aaa-4c86-928c-09e2f6d8f8b4-kube-api-access-ppddl\") pod \"node-ca-slfll\" (UID: \"b16b5696-7aaa-4c86-928c-09e2f6d8f8b4\") " pod="openshift-image-registry/node-ca-slfll" Aug 04 09:40:33.288247 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.288225 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twxtp\" (UniqueName: \"kubernetes.io/projected/f192c223-2f86-4ee1-836a-60af7de6824a-kube-api-access-twxtp\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:33.289817 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.289774 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tvpw\" (UniqueName: \"kubernetes.io/projected/f843d0d9-03b6-40f7-ae3c-a5d5b641168e-kube-api-access-6tvpw\") pod \"ovnkube-node-rb78b\" (UID: \"f843d0d9-03b6-40f7-ae3c-a5d5b641168e\") " pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.290633 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.290597 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85nnq\" (UniqueName: \"kubernetes.io/projected/6309670e-2dff-4216-b018-1ed218e5da34-kube-api-access-85nnq\") pod \"multus-additional-cni-plugins-qmwhl\" (UID: \"6309670e-2dff-4216-b018-1ed218e5da34\") " pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.337630 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.337582 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" event={"ID":"3abd502c334bfa63fadf295cd4cdea9c","Type":"ContainerStarted","Data":"b9395516e7232c7349afec35976dc163abaa84f84f0ea46419a18b399f77f7ea"} Aug 04 09:40:33.338631 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.338603 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-10.ec2.internal" event={"ID":"499f0d85eada0e49ab159e031fad0580","Type":"ContainerStarted","Data":"00cc5763499b28246527c8ac7c68e2eb009fb8343ff460f8c8f01697b5cca2a3"} Aug 04 09:40:33.377934 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.377845 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-kubernetes\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.377934 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.377898 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-sys\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.377984 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-kubernetes\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378011 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-lib-modules\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378040 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-host\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378067 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-sysconfig\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378068 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-sys\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378090 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-modprobe-d\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378114 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-var-lib-kubelet\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378118 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-host\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378158 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-sysconfig\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378177 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-var-lib-kubelet\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378187 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-run\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378224 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-sysctl-conf\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378220 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-lib-modules\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378255 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8612b3d8-d52c-4b28-96b1-c9202698e69d-tmp-dir\") pod \"node-resolver-vkqkd\" (UID: \"8612b3d8-d52c-4b28-96b1-c9202698e69d\") " pod="openshift-dns/node-resolver-vkqkd" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378278 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-modprobe-d\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378295 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-tmp\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378296 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-run\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378325 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9svj\" (UniqueName: \"kubernetes.io/projected/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-kube-api-access-d9svj\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378350 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8612b3d8-d52c-4b28-96b1-c9202698e69d-hosts-file\") pod \"node-resolver-vkqkd\" (UID: \"8612b3d8-d52c-4b28-96b1-c9202698e69d\") " pod="openshift-dns/node-resolver-vkqkd" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378372 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-sysctl-conf\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378376 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-sysctl-d\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378418 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8612b3d8-d52c-4b28-96b1-c9202698e69d-hosts-file\") pod \"node-resolver-vkqkd\" (UID: \"8612b3d8-d52c-4b28-96b1-c9202698e69d\") " pod="openshift-dns/node-resolver-vkqkd" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378419 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spkkj\" (UniqueName: \"kubernetes.io/projected/8612b3d8-d52c-4b28-96b1-c9202698e69d-kube-api-access-spkkj\") pod \"node-resolver-vkqkd\" (UID: \"8612b3d8-d52c-4b28-96b1-c9202698e69d\") " pod="openshift-dns/node-resolver-vkqkd" Aug 04 09:40:33.378453 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378457 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-systemd\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.379143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378483 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-tuned\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.379143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378506 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-sysctl-d\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.379143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378559 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-systemd\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.379143 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.378561 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8612b3d8-d52c-4b28-96b1-c9202698e69d-tmp-dir\") pod \"node-resolver-vkqkd\" (UID: \"8612b3d8-d52c-4b28-96b1-c9202698e69d\") " pod="openshift-dns/node-resolver-vkqkd" Aug 04 09:40:33.380640 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.380619 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-tmp\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.380747 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.380667 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-etc-tuned\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.387181 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.387157 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9svj\" (UniqueName: \"kubernetes.io/projected/d81b4d59-84f7-4bda-ba50-6fa8f528acb6-kube-api-access-d9svj\") pod \"tuned-24dll\" (UID: \"d81b4d59-84f7-4bda-ba50-6fa8f528acb6\") " pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.387311 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.387242 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spkkj\" (UniqueName: \"kubernetes.io/projected/8612b3d8-d52c-4b28-96b1-c9202698e69d-kube-api-access-spkkj\") pod \"node-resolver-vkqkd\" (UID: \"8612b3d8-d52c-4b28-96b1-c9202698e69d\") " pod="openshift-dns/node-resolver-vkqkd" Aug 04 09:40:33.400366 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.400331 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:40:33.446681 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.446645 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vczmz" Aug 04 09:40:33.454980 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.454948 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" Aug 04 09:40:33.464838 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.464809 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:40:33.471789 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.471765 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-slfll" Aug 04 09:40:33.480834 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.480811 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:33.488674 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.488649 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t6qlp" Aug 04 09:40:33.496484 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.496463 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" Aug 04 09:40:33.505307 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.505284 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-24dll" Aug 04 09:40:33.510007 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.509984 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vkqkd" Aug 04 09:40:33.782142 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.782055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:33.782142 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.782099 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:33.782360 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.782229 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:33.782360 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.782293 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs podName:f192c223-2f86-4ee1-836a-60af7de6824a nodeName:}" failed. No retries permitted until 2026-08-04 09:40:34.782276125 +0000 UTC m=+3.973568612 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs") pod "network-metrics-daemon-pcs94" (UID: "f192c223-2f86-4ee1-836a-60af7de6824a") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:33.782360 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.782232 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:33.782513 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.782376 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret podName:a2004042-19e9-46e9-8379-ad03ca29d486 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:34.782357739 +0000 UTC m=+3.973650218 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret") pod "global-pull-secret-syncer-g6s9t" (UID: "a2004042-19e9-46e9-8379-ad03ca29d486") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:33.882700 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:33.882655 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tj67\" (UniqueName: \"kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67\") pod \"network-check-target-c6bvn\" (UID: \"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7\") " pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:33.882886 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.882851 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:33.882886 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.882874 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:33.882886 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.882887 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6tj67 for pod openshift-network-diagnostics/network-check-target-c6bvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:33.883040 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:33.882954 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67 podName:9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:34.882935994 +0000 UTC m=+4.074228483 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tj67" (UniqueName: "kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67") pod "network-check-target-c6bvn" (UID: "9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:33.962403 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:33.962375 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45e3d1c_b6c9_4f3a_91e4_3f7ecbf43222.slice/crio-381a25f6bd32d62df77765201d666c44139a827d4d4da20ef0c09ca6399a3a0b WatchSource:0}: Error finding container 381a25f6bd32d62df77765201d666c44139a827d4d4da20ef0c09ca6399a3a0b: Status 404 returned error can't find the container with id 381a25f6bd32d62df77765201d666c44139a827d4d4da20ef0c09ca6399a3a0b Aug 04 09:40:33.964222 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:33.964176 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd81b4d59_84f7_4bda_ba50_6fa8f528acb6.slice/crio-39a7fab39b14a01b0af3cc952a30f6e705c3b745236565a6d6272f8441479d2c WatchSource:0}: Error finding container 39a7fab39b14a01b0af3cc952a30f6e705c3b745236565a6d6272f8441479d2c: Status 404 returned error can't find the container with id 39a7fab39b14a01b0af3cc952a30f6e705c3b745236565a6d6272f8441479d2c Aug 04 09:40:33.965546 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:33.965522 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb16b5696_7aaa_4c86_928c_09e2f6d8f8b4.slice/crio-3e04f186972fe493b68de74686a0da71d736cd91b6de8d9b8827b56dafb1c317 WatchSource:0}: Error finding container 3e04f186972fe493b68de74686a0da71d736cd91b6de8d9b8827b56dafb1c317: Status 404 returned error can't find the container with id 3e04f186972fe493b68de74686a0da71d736cd91b6de8d9b8827b56dafb1c317 Aug 04 09:40:33.968912 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:33.968887 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf843d0d9_03b6_40f7_ae3c_a5d5b641168e.slice/crio-cb548bdcad2826dd676f70afdaaced60bb4979290b6e2b57f81ffefa841e6680 WatchSource:0}: Error finding container cb548bdcad2826dd676f70afdaaced60bb4979290b6e2b57f81ffefa841e6680: Status 404 returned error can't find the container with id cb548bdcad2826dd676f70afdaaced60bb4979290b6e2b57f81ffefa841e6680 Aug 04 09:40:33.969546 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:33.969529 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod250d4003_cf01_4a28_9b5e_128d047c2260.slice/crio-0069d567749eedf6a44c7feb4c4e9e7f54c674a833ffe5f7b8f82bd4dde8d375 WatchSource:0}: Error finding container 0069d567749eedf6a44c7feb4c4e9e7f54c674a833ffe5f7b8f82bd4dde8d375: Status 404 returned error can't find the container with id 0069d567749eedf6a44c7feb4c4e9e7f54c674a833ffe5f7b8f82bd4dde8d375 Aug 04 09:40:33.970618 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:33.970593 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8612b3d8_d52c_4b28_96b1_c9202698e69d.slice/crio-4f8d140de1f1db66cac170924d50ddaa6a15b9f1e8e32c52cdc114ac653034f8 WatchSource:0}: Error finding container 4f8d140de1f1db66cac170924d50ddaa6a15b9f1e8e32c52cdc114ac653034f8: Status 404 returned error can't find the container with id 4f8d140de1f1db66cac170924d50ddaa6a15b9f1e8e32c52cdc114ac653034f8 Aug 04 09:40:33.971081 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:33.971060 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75063389_386c_4849_9baf_c001a3c3f9d2.slice/crio-d98aed6bb8fc3178f3ad3f3b63e31ac8a56f6febb67f2176e1d77c93f6d5f8ea WatchSource:0}: Error finding container d98aed6bb8fc3178f3ad3f3b63e31ac8a56f6febb67f2176e1d77c93f6d5f8ea: Status 404 returned error can't find the container with id d98aed6bb8fc3178f3ad3f3b63e31ac8a56f6febb67f2176e1d77c93f6d5f8ea Aug 04 09:40:33.972766 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:33.972711 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed15a75_90cc_42bc_831e_e343ac873cf6.slice/crio-affc03d02bc2786db833c0cc7668d9af9468e00edd854dba0834a2a5cc2fc93a WatchSource:0}: Error finding container affc03d02bc2786db833c0cc7668d9af9468e00edd854dba0834a2a5cc2fc93a: Status 404 returned error can't find the container with id affc03d02bc2786db833c0cc7668d9af9468e00edd854dba0834a2a5cc2fc93a Aug 04 09:40:33.973637 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:40:33.973617 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6309670e_2dff_4216_b018_1ed218e5da34.slice/crio-85c8faadd3c1c5d64f5a04203007480810b246552b99a9fb3da08bed00c654c9 WatchSource:0}: Error finding container 85c8faadd3c1c5d64f5a04203007480810b246552b99a9fb3da08bed00c654c9: Status 404 returned error can't find the container with id 85c8faadd3c1c5d64f5a04203007480810b246552b99a9fb3da08bed00c654c9 Aug 04 09:40:34.279240 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.279175 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 09:35:32 +0000 UTC" deadline="2028-01-21 19:10:14.222298865 +0000 UTC" Aug 04 09:40:34.279240 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.279225 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12849h29m39.943076533s" Aug 04 09:40:34.332226 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.332074 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:34.332226 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.332124 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:34.332226 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:34.332212 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:34.332453 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:34.332365 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:34.342568 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.342472 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" event={"ID":"6309670e-2dff-4216-b018-1ed218e5da34","Type":"ContainerStarted","Data":"85c8faadd3c1c5d64f5a04203007480810b246552b99a9fb3da08bed00c654c9"} Aug 04 09:40:34.343864 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.343824 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lbmxb" event={"ID":"75063389-386c-4849-9baf-c001a3c3f9d2","Type":"ContainerStarted","Data":"d98aed6bb8fc3178f3ad3f3b63e31ac8a56f6febb67f2176e1d77c93f6d5f8ea"} Aug 04 09:40:34.345605 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.345580 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vczmz" event={"ID":"250d4003-cf01-4a28-9b5e-128d047c2260","Type":"ContainerStarted","Data":"0069d567749eedf6a44c7feb4c4e9e7f54c674a833ffe5f7b8f82bd4dde8d375"} Aug 04 09:40:34.347191 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.347161 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-slfll" event={"ID":"b16b5696-7aaa-4c86-928c-09e2f6d8f8b4","Type":"ContainerStarted","Data":"3e04f186972fe493b68de74686a0da71d736cd91b6de8d9b8827b56dafb1c317"} Aug 04 09:40:34.348630 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.348607 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-24dll" event={"ID":"d81b4d59-84f7-4bda-ba50-6fa8f528acb6","Type":"ContainerStarted","Data":"39a7fab39b14a01b0af3cc952a30f6e705c3b745236565a6d6272f8441479d2c"} Aug 04 09:40:34.349868 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.349835 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t6qlp" event={"ID":"f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222","Type":"ContainerStarted","Data":"381a25f6bd32d62df77765201d666c44139a827d4d4da20ef0c09ca6399a3a0b"} Aug 04 09:40:34.352541 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.352358 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-10.ec2.internal" event={"ID":"499f0d85eada0e49ab159e031fad0580","Type":"ContainerStarted","Data":"637abcbb74d97ef1a633194e730f76e6b2410811d25671140838f82bfb9b2f18"} Aug 04 09:40:34.353702 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.353674 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" event={"ID":"fed15a75-90cc-42bc-831e-e343ac873cf6","Type":"ContainerStarted","Data":"affc03d02bc2786db833c0cc7668d9af9468e00edd854dba0834a2a5cc2fc93a"} Aug 04 09:40:34.354996 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.354969 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vkqkd" event={"ID":"8612b3d8-d52c-4b28-96b1-c9202698e69d","Type":"ContainerStarted","Data":"4f8d140de1f1db66cac170924d50ddaa6a15b9f1e8e32c52cdc114ac653034f8"} Aug 04 09:40:34.356486 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.356459 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" event={"ID":"f843d0d9-03b6-40f7-ae3c-a5d5b641168e","Type":"ContainerStarted","Data":"cb548bdcad2826dd676f70afdaaced60bb4979290b6e2b57f81ffefa841e6680"} Aug 04 09:40:34.373629 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.373482 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-10.ec2.internal" podStartSLOduration=2.37346103 podStartE2EDuration="2.37346103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:34.373039462 +0000 UTC m=+3.564331952" watchObservedRunningTime="2026-08-04 09:40:34.37346103 +0000 UTC m=+3.564753525" Aug 04 09:40:34.790357 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.790265 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:34.790357 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.790318 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:34.790568 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:34.790445 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:34.790568 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:34.790509 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs podName:f192c223-2f86-4ee1-836a-60af7de6824a nodeName:}" failed. No retries permitted until 2026-08-04 09:40:36.790491856 +0000 UTC m=+5.981784331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs") pod "network-metrics-daemon-pcs94" (UID: "f192c223-2f86-4ee1-836a-60af7de6824a") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:34.790988 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:34.790845 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:34.790988 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:34.790902 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret podName:a2004042-19e9-46e9-8379-ad03ca29d486 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:36.790887017 +0000 UTC m=+5.982179492 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret") pod "global-pull-secret-syncer-g6s9t" (UID: "a2004042-19e9-46e9-8379-ad03ca29d486") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:34.891353 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:34.891319 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tj67\" (UniqueName: \"kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67\") pod \"network-check-target-c6bvn\" (UID: \"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7\") " pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:34.891532 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:34.891455 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:34.891532 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:34.891468 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:34.891532 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:34.891477 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6tj67 for pod openshift-network-diagnostics/network-check-target-c6bvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:34.891532 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:34.891522 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67 podName:9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:36.89150998 +0000 UTC m=+6.082802453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tj67" (UniqueName: "kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67") pod "network-check-target-c6bvn" (UID: "9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:35.339657 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:35.338951 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:35.339657 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:35.339319 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:36.334031 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:36.332976 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:36.334031 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:36.333122 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:36.334031 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:36.333713 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:36.334031 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:36.333893 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:36.814934 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:36.814844 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:36.814934 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:36.814903 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:36.815438 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:36.815046 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:36.815438 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:36.815106 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs podName:f192c223-2f86-4ee1-836a-60af7de6824a nodeName:}" failed. No retries permitted until 2026-08-04 09:40:40.815087461 +0000 UTC m=+10.006379939 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs") pod "network-metrics-daemon-pcs94" (UID: "f192c223-2f86-4ee1-836a-60af7de6824a") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:36.815545 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:36.815495 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:36.815545 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:36.815540 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret podName:a2004042-19e9-46e9-8379-ad03ca29d486 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:40.815524489 +0000 UTC m=+10.006816981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret") pod "global-pull-secret-syncer-g6s9t" (UID: "a2004042-19e9-46e9-8379-ad03ca29d486") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:36.916106 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:36.915877 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tj67\" (UniqueName: \"kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67\") pod \"network-check-target-c6bvn\" (UID: \"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7\") " pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:36.916302 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:36.916130 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:36.916302 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:36.916151 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:36.916302 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:36.916163 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6tj67 for pod openshift-network-diagnostics/network-check-target-c6bvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:36.916302 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:36.916225 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67 podName:9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:40.916206861 +0000 UTC m=+10.107499338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tj67" (UniqueName: "kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67") pod "network-check-target-c6bvn" (UID: "9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:37.333470 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:37.332914 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:37.333470 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:37.333085 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:38.332893 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:38.332863 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:38.333348 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:38.332916 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:38.333348 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:38.332997 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:38.333348 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:38.333092 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:39.335738 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:39.335705 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:39.336160 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:39.335846 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:40.332681 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:40.332647 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:40.332846 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:40.332693 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:40.332846 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:40.332796 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:40.332964 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:40.332930 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:40.849987 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:40.849943 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:40.850426 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:40.849993 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:40.850426 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:40.850111 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:40.850426 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:40.850175 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs podName:f192c223-2f86-4ee1-836a-60af7de6824a nodeName:}" failed. No retries permitted until 2026-08-04 09:40:48.850156704 +0000 UTC m=+18.041449191 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs") pod "network-metrics-daemon-pcs94" (UID: "f192c223-2f86-4ee1-836a-60af7de6824a") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:40.850426 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:40.850111 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:40.850426 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:40.850250 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret podName:a2004042-19e9-46e9-8379-ad03ca29d486 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:48.85023266 +0000 UTC m=+18.041525147 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret") pod "global-pull-secret-syncer-g6s9t" (UID: "a2004042-19e9-46e9-8379-ad03ca29d486") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:40.951227 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:40.951026 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tj67\" (UniqueName: \"kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67\") pod \"network-check-target-c6bvn\" (UID: \"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7\") " pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:40.951227 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:40.951183 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:40.951227 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:40.951201 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:40.951227 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:40.951213 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6tj67 for pod openshift-network-diagnostics/network-check-target-c6bvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:40.951551 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:40.951276 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67 podName:9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:48.951258082 +0000 UTC m=+18.142550556 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tj67" (UniqueName: "kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67") pod "network-check-target-c6bvn" (UID: "9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:41.333813 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:41.333740 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:41.333963 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:41.333866 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:42.332863 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:42.332528 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:42.332863 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:42.332543 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:42.332863 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:42.332666 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:42.333410 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:42.333077 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:43.332472 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:43.332440 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:43.332634 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:43.332559 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:44.332148 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:44.332120 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:44.332499 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:44.332122 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:44.332499 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:44.332212 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:44.332499 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:44.332287 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:45.332271 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:45.332194 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:45.332719 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:45.332332 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:46.332754 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:46.332708 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:46.333204 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:46.332709 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:46.333204 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:46.332859 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:46.333204 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:46.332897 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:47.332523 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:47.332486 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:47.332714 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:47.332615 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:48.332712 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:48.332678 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:48.333233 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:48.332681 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:48.333233 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:48.332812 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:48.333233 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:48.332907 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:48.911202 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:48.911159 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:48.911385 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:48.911210 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:48.911385 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:48.911332 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:48.911385 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:48.911343 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:48.911533 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:48.911411 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret podName:a2004042-19e9-46e9-8379-ad03ca29d486 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:04.911388734 +0000 UTC m=+34.102681210 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret") pod "global-pull-secret-syncer-g6s9t" (UID: "a2004042-19e9-46e9-8379-ad03ca29d486") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:48.911533 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:48.911433 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs podName:f192c223-2f86-4ee1-836a-60af7de6824a nodeName:}" failed. No retries permitted until 2026-08-04 09:41:04.911422708 +0000 UTC m=+34.102715182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs") pod "network-metrics-daemon-pcs94" (UID: "f192c223-2f86-4ee1-836a-60af7de6824a") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:49.012068 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:49.012032 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tj67\" (UniqueName: \"kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67\") pod \"network-check-target-c6bvn\" (UID: \"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7\") " pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:49.012242 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:49.012155 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:49.012242 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:49.012177 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:49.012242 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:49.012189 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6tj67 for pod openshift-network-diagnostics/network-check-target-c6bvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:49.012390 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:49.012248 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67 podName:9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:05.012228369 +0000 UTC m=+34.203520855 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tj67" (UniqueName: "kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67") pod "network-check-target-c6bvn" (UID: "9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:49.332482 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:49.332395 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:49.332637 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:49.332534 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:50.332338 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:50.332310 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:50.332749 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:50.332451 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:50.332749 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:50.332540 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:50.332749 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:50.332659 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:51.332744 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:51.332702 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:51.333121 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:51.332848 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:52.332664 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:52.332637 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:52.332802 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:52.332743 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:52.332802 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:52.332787 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:52.333062 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:52.332868 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:53.333101 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.332882 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:53.333915 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:53.333140 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:53.393275 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.393245 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lbmxb" event={"ID":"75063389-386c-4849-9baf-c001a3c3f9d2","Type":"ContainerStarted","Data":"2d8cd2a3d4c0c8756dfe015028b4256f7b37f5a12fc95449349f188a77ab71aa"} Aug 04 09:40:53.394568 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.394542 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vczmz" event={"ID":"250d4003-cf01-4a28-9b5e-128d047c2260","Type":"ContainerStarted","Data":"1fb30086b4a05b6672326a25b3a9b7541306d46882e4b2f352494dfa955739bf"} Aug 04 09:40:53.395843 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.395821 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-slfll" event={"ID":"b16b5696-7aaa-4c86-928c-09e2f6d8f8b4","Type":"ContainerStarted","Data":"d67aa1f7be3fd0b072c52cbb1ca03aded3008498e379daf566fbd32cc1cf3811"} Aug 04 09:40:53.399409 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.399381 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-24dll" event={"ID":"d81b4d59-84f7-4bda-ba50-6fa8f528acb6","Type":"ContainerStarted","Data":"1aa41e8ec0bdefd36a8f0a4255fc4219fe1ec16a02b2f4b9f3a1ac8859376a16"} Aug 04 09:40:53.400649 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.400625 2575 generic.go:356] "Generic (PLEG): container finished" podID="3abd502c334bfa63fadf295cd4cdea9c" containerID="6c87c959189989a0218121828cb0e204cd289c38691e1ea6e88455eb8f8934e9" exitCode=0 Aug 04 09:40:53.400765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.400691 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" event={"ID":"3abd502c334bfa63fadf295cd4cdea9c","Type":"ContainerDied","Data":"6c87c959189989a0218121828cb0e204cd289c38691e1ea6e88455eb8f8934e9"} Aug 04 09:40:53.402093 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.402066 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" event={"ID":"fed15a75-90cc-42bc-831e-e343ac873cf6","Type":"ContainerStarted","Data":"cee144d0328b8a95d6d6252fd2458e6079a1f516271b374c35176492b95d4c52"} Aug 04 09:40:53.403310 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.403291 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vkqkd" event={"ID":"8612b3d8-d52c-4b28-96b1-c9202698e69d","Type":"ContainerStarted","Data":"a90811530ada73623888e425121a5462623f1b9b31fe7f53b30800bd21b4425c"} Aug 04 09:40:53.405896 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.405875 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" event={"ID":"f843d0d9-03b6-40f7-ae3c-a5d5b641168e","Type":"ContainerStarted","Data":"57936080ec5e883d5208f2d0593e2afbe71f7f15a7ee7515957b22d482fe9fa0"} Aug 04 09:40:53.405985 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.405935 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" event={"ID":"f843d0d9-03b6-40f7-ae3c-a5d5b641168e","Type":"ContainerStarted","Data":"e853dd69af67245d1b239421b3210e768fa313a05459fdd5b977e215e99ed14a"} Aug 04 09:40:53.405985 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.405977 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" event={"ID":"f843d0d9-03b6-40f7-ae3c-a5d5b641168e","Type":"ContainerStarted","Data":"afbc40349bd7f7d832cdab43d7ef4bb3dc6b46485e3f82783c6af5e11f749344"} Aug 04 09:40:53.406057 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.406011 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" event={"ID":"f843d0d9-03b6-40f7-ae3c-a5d5b641168e","Type":"ContainerStarted","Data":"25b884baa67216d97cd5f5a68740de2f9e4f10058cd93f20664b67a08b5903ba"} Aug 04 09:40:53.406057 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.406051 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" event={"ID":"f843d0d9-03b6-40f7-ae3c-a5d5b641168e","Type":"ContainerStarted","Data":"eb64a334eb7c9f1a59ff4b9b481223058f7d5ba2c2d2d90b824dd6c3211886d7"} Aug 04 09:40:53.406129 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.406091 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" event={"ID":"f843d0d9-03b6-40f7-ae3c-a5d5b641168e","Type":"ContainerStarted","Data":"1acd320d3e0465a357ec94758a818940ef437785de53df513f743a6e0ddc9a07"} Aug 04 09:40:53.407277 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.407253 2575 generic.go:356] "Generic (PLEG): container finished" podID="6309670e-2dff-4216-b018-1ed218e5da34" containerID="ce7acbdcd37d918bef8591008700dd1329c0f8cee671c4ad9e22567222c679c7" exitCode=0 Aug 04 09:40:53.407360 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.407286 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" event={"ID":"6309670e-2dff-4216-b018-1ed218e5da34","Type":"ContainerDied","Data":"ce7acbdcd37d918bef8591008700dd1329c0f8cee671c4ad9e22567222c679c7"} Aug 04 09:40:53.412458 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.412424 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lbmxb" podStartSLOduration=3.9914495690000003 podStartE2EDuration="22.412408714s" totalImagesPullingTime="18.420959145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:53.412299135 +0000 UTC m=+22.603591628" watchObservedRunningTime="2026-08-04 09:40:53.412408714 +0000 UTC m=+22.603701208" Aug 04 09:40:53.442304 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.442275 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vkqkd" podStartSLOduration=3.982023668 podStartE2EDuration="22.442263469s" totalImagesPullingTime="18.460239801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:53.441675223 +0000 UTC m=+22.632967741" watchObservedRunningTime="2026-08-04 09:40:53.442263469 +0000 UTC m=+22.633555962" Aug 04 09:40:53.528956 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.528753 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-slfll" podStartSLOduration=4.10260332 podStartE2EDuration="22.52871258s" totalImagesPullingTime="18.42610926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:53.501849858 +0000 UTC m=+22.693142349" watchObservedRunningTime="2026-08-04 09:40:53.52871258 +0000 UTC m=+22.720005076" Aug 04 09:40:53.596723 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:53.596646 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vczmz" podStartSLOduration=3.921277069 podStartE2EDuration="22.596634157s" totalImagesPullingTime="18.675357088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:53.537874388 +0000 UTC m=+22.729166882" watchObservedRunningTime="2026-08-04 09:40:53.596634157 +0000 UTC m=+22.787926648" Aug 04 09:40:54.154952 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:54.154918 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:40:54.155648 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:54.155624 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:40:54.172042 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:54.171999 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-24dll" podStartSLOduration=4.745040997 podStartE2EDuration="23.171981369s" totalImagesPullingTime="18.426940372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:53.596719406 +0000 UTC m=+22.788011900" watchObservedRunningTime="2026-08-04 09:40:54.171981369 +0000 UTC m=+23.363273870" Aug 04 09:40:54.332503 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:54.332471 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:54.332503 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:54.332505 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:54.332761 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:54.332587 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:54.332761 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:54.332708 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:54.411642 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:54.411563 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t6qlp" event={"ID":"f45e3d1c-b6c9-4f3a-91e4-3f7ecbf43222","Type":"ContainerStarted","Data":"69d01c8b7a84208b9a5327b55e0e4fe3d31d82804b8be52293d110a297b6bce4"} Aug 04 09:40:54.413559 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:54.413526 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" event={"ID":"3abd502c334bfa63fadf295cd4cdea9c","Type":"ContainerStarted","Data":"a5403230746c3c67592b51fe8ef8e4adc24b9e09562140394d595924c9006ab6"} Aug 04 09:40:54.429320 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:54.429277 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t6qlp" podStartSLOduration=4.957752921 podStartE2EDuration="23.429264924s" totalImagesPullingTime="18.471512003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:54.428700927 +0000 UTC m=+23.619993432" watchObservedRunningTime="2026-08-04 09:40:54.429264924 +0000 UTC m=+23.620557419" Aug 04 09:40:55.333669 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:55.333422 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:55.333669 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:55.333554 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:55.414004 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:55.413768 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 04 09:40:55.417620 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:55.417595 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" event={"ID":"fed15a75-90cc-42bc-831e-e343ac873cf6","Type":"ContainerStarted","Data":"e5c406c068ea9d05e17c3d1f849cfdf378b9e8b122131efa094179bb248f85fd"} Aug 04 09:40:55.420373 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:55.420291 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" event={"ID":"f843d0d9-03b6-40f7-ae3c-a5d5b641168e","Type":"ContainerStarted","Data":"afd7efa6db90a78a10ee68dc03c5a0526ca10f3fc1cd79e7daae59efb224e83f"} Aug 04 09:40:55.420683 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:55.420583 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 04 09:40:56.229228 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:56.229137 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T09:40:55.413788089Z","UUID":"1e122e4b-648d-41b8-87ea-5dac98949069","Handler":null,"Name":"","Endpoint":""} Aug 04 09:40:56.230688 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:56.230666 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 04 09:40:56.230845 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:56.230710 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 04 09:40:56.332538 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:56.332506 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:56.332679 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:56.332517 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:56.332679 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:56.332629 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:56.332796 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:56.332688 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:57.332773 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:57.332741 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:57.333242 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:57.332868 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:40:58.332397 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.332237 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:40:58.332521 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.332240 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:40:58.332521 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:58.332458 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:40:58.332600 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:58.332535 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:40:58.428298 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.428236 2575 generic.go:356] "Generic (PLEG): container finished" podID="6309670e-2dff-4216-b018-1ed218e5da34" containerID="fcfb13a8230cff4a06f8f322859aa2a74f444c1fb22f168b6ad5f39c921bd711" exitCode=0 Aug 04 09:40:58.429148 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.428323 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" event={"ID":"6309670e-2dff-4216-b018-1ed218e5da34","Type":"ContainerDied","Data":"fcfb13a8230cff4a06f8f322859aa2a74f444c1fb22f168b6ad5f39c921bd711"} Aug 04 09:40:58.430581 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.430561 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" event={"ID":"fed15a75-90cc-42bc-831e-e343ac873cf6","Type":"ContainerStarted","Data":"7be666a59e65ef1fab983b428bae34d3fbfb619de625f92c0467e52f721246f6"} Aug 04 09:40:58.434546 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.434525 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" event={"ID":"f843d0d9-03b6-40f7-ae3c-a5d5b641168e","Type":"ContainerStarted","Data":"f938a36049602b84211298f49862651e2a69e636f6b55cbd897785dad80908dc"} Aug 04 09:40:58.434777 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.434755 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:58.434859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.434818 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:58.434859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.434855 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:58.449874 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.449852 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:58.450084 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.450070 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:40:58.453022 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.452981 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-10.ec2.internal" podStartSLOduration=26.452965985 podStartE2EDuration="26.452965985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:54.444836123 +0000 UTC m=+23.636128618" watchObservedRunningTime="2026-08-04 09:40:58.452965985 +0000 UTC m=+27.644258475" Aug 04 09:40:58.470248 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.470218 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5942" podStartSLOduration=4.359304378 podStartE2EDuration="27.470207685s" totalImagesPullingTime="23.110903307s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:58.469958278 +0000 UTC m=+27.661250773" watchObservedRunningTime="2026-08-04 09:40:58.470207685 +0000 UTC m=+27.661500179" Aug 04 09:40:58.495885 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:58.495854 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" podStartSLOduration=9.007362975 podStartE2EDuration="27.495843637s" totalImagesPullingTime="18.488480662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:58.495539629 +0000 UTC m=+27.686832124" watchObservedRunningTime="2026-08-04 09:40:58.495843637 +0000 UTC m=+27.687136131" Aug 04 09:40:59.332577 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:40:59.332544 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:40:59.332716 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:40:59.332656 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:41:00.115875 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:00.115681 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g6s9t"] Aug 04 09:41:00.116221 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:00.115966 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:41:00.116221 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:00.116057 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:41:00.119816 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:00.119706 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pcs94"] Aug 04 09:41:00.119939 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:00.119835 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:41:00.120002 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:00.119948 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:41:00.120394 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:00.120370 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c6bvn"] Aug 04 09:41:00.120489 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:00.120459 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:41:00.120545 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:00.120526 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:41:00.441203 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:00.441175 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" event={"ID":"6309670e-2dff-4216-b018-1ed218e5da34","Type":"ContainerStarted","Data":"1446d71318e6de1ab6ea7839341fd8d3b7e20b33f7667c50f3bf06a7f5871ef9"} Aug 04 09:41:01.333401 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:01.333369 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:41:01.333850 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:01.333467 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:41:01.445676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:01.445648 2575 generic.go:356] "Generic (PLEG): container finished" podID="6309670e-2dff-4216-b018-1ed218e5da34" containerID="1446d71318e6de1ab6ea7839341fd8d3b7e20b33f7667c50f3bf06a7f5871ef9" exitCode=0 Aug 04 09:41:01.445816 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:01.445686 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" event={"ID":"6309670e-2dff-4216-b018-1ed218e5da34","Type":"ContainerDied","Data":"1446d71318e6de1ab6ea7839341fd8d3b7e20b33f7667c50f3bf06a7f5871ef9"} Aug 04 09:41:02.332756 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:02.332675 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:41:02.332933 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:02.332799 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:41:02.332933 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:02.332855 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:41:02.333061 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:02.332986 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:41:03.332842 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:03.332814 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:41:03.333152 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:03.332969 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6s9t" podUID="a2004042-19e9-46e9-8379-ad03ca29d486" Aug 04 09:41:03.349246 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:03.349221 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:41:03.349356 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:03.349321 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 04 09:41:03.349823 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:03.349797 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lbmxb" Aug 04 09:41:03.452303 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:03.452282 2575 generic.go:356] "Generic (PLEG): container finished" podID="6309670e-2dff-4216-b018-1ed218e5da34" containerID="0f7077c2c101377bb4e0d14222771ca3fc7bc45c783806eb17468fe7ec1e167d" exitCode=0 Aug 04 09:41:03.452396 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:03.452349 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" event={"ID":"6309670e-2dff-4216-b018-1ed218e5da34","Type":"ContainerDied","Data":"0f7077c2c101377bb4e0d14222771ca3fc7bc45c783806eb17468fe7ec1e167d"} Aug 04 09:41:04.332720 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:04.332689 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:41:04.332902 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:04.332805 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6bvn" podUID="9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7" Aug 04 09:41:04.332902 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:04.332863 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:41:04.333297 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:04.332995 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pcs94" podUID="f192c223-2f86-4ee1-836a-60af7de6824a" Aug 04 09:41:04.936895 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:04.936859 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:41:04.936895 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:04.936899 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:41:04.937110 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:04.937027 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:41:04.937110 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:04.937074 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:41:04.937110 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:04.937090 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret podName:a2004042-19e9-46e9-8379-ad03ca29d486 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:36.937073134 +0000 UTC m=+66.128365613 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret") pod "global-pull-secret-syncer-g6s9t" (UID: "a2004042-19e9-46e9-8379-ad03ca29d486") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:41:04.937228 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:04.937124 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs podName:f192c223-2f86-4ee1-836a-60af7de6824a nodeName:}" failed. No retries permitted until 2026-08-04 09:41:36.937106769 +0000 UTC m=+66.128399244 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs") pod "network-metrics-daemon-pcs94" (UID: "f192c223-2f86-4ee1-836a-60af7de6824a") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:41:05.037929 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.037897 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tj67\" (UniqueName: \"kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67\") pod \"network-check-target-c6bvn\" (UID: \"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7\") " pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:41:05.038095 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:05.038064 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:41:05.038095 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:05.038083 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:41:05.038095 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:05.038092 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6tj67 for pod openshift-network-diagnostics/network-check-target-c6bvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:41:05.038257 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:05.038145 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67 podName:9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:37.038129176 +0000 UTC m=+66.229421672 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tj67" (UniqueName: "kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67") pod "network-check-target-c6bvn" (UID: "9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:41:05.117001 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.116979 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-10.ec2.internal" event="NodeReady" Aug 04 09:41:05.117138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.117124 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 04 09:41:05.176176 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.176147 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-47flm"] Aug 04 09:41:05.178200 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.178180 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.178828 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.178811 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-htz5h"] Aug 04 09:41:05.180387 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.180367 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-htz5h" Aug 04 09:41:05.187416 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.187368 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 04 09:41:05.189616 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.189557 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 04 09:41:05.189616 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.189581 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 04 09:41:05.189789 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.189648 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 04 09:41:05.189863 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.189818 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 04 09:41:05.189943 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.189914 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-psh6m\"" Aug 04 09:41:05.190006 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.189947 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rgp27\"" Aug 04 09:41:05.195269 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.195240 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-47flm"] Aug 04 09:41:05.199853 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.199834 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-htz5h"] Aug 04 09:41:05.332145 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.332101 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:41:05.334759 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.334656 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 09:41:05.339017 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.338970 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh2rr\" (UniqueName: \"kubernetes.io/projected/6c66c8c6-437d-4958-b76b-544874765c73-kube-api-access-vh2rr\") pod \"ingress-canary-htz5h\" (UID: \"6c66c8c6-437d-4958-b76b-544874765c73\") " pod="openshift-ingress-canary/ingress-canary-htz5h" Aug 04 09:41:05.339094 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.339014 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.339094 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.339041 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6adbcf4-6061-4e4c-b52f-092d5a911104-config-volume\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.339094 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.339061 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c66c8c6-437d-4958-b76b-544874765c73-cert\") pod \"ingress-canary-htz5h\" (UID: \"6c66c8c6-437d-4958-b76b-544874765c73\") " pod="openshift-ingress-canary/ingress-canary-htz5h" Aug 04 09:41:05.339246 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.339158 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6adbcf4-6061-4e4c-b52f-092d5a911104-tmp-dir\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.339246 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.339235 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vsrr\" (UniqueName: \"kubernetes.io/projected/e6adbcf4-6061-4e4c-b52f-092d5a911104-kube-api-access-2vsrr\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.440523 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.440457 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vsrr\" (UniqueName: \"kubernetes.io/projected/e6adbcf4-6061-4e4c-b52f-092d5a911104-kube-api-access-2vsrr\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.440523 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.440501 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh2rr\" (UniqueName: \"kubernetes.io/projected/6c66c8c6-437d-4958-b76b-544874765c73-kube-api-access-vh2rr\") pod \"ingress-canary-htz5h\" (UID: \"6c66c8c6-437d-4958-b76b-544874765c73\") " pod="openshift-ingress-canary/ingress-canary-htz5h" Aug 04 09:41:05.440680 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.440528 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.440680 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.440553 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6adbcf4-6061-4e4c-b52f-092d5a911104-config-volume\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.440680 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.440578 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c66c8c6-437d-4958-b76b-544874765c73-cert\") pod \"ingress-canary-htz5h\" (UID: \"6c66c8c6-437d-4958-b76b-544874765c73\") " pod="openshift-ingress-canary/ingress-canary-htz5h" Aug 04 09:41:05.440680 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.440624 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6adbcf4-6061-4e4c-b52f-092d5a911104-tmp-dir\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.440901 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:05.440696 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:05.440901 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:05.440808 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls podName:e6adbcf4-6061-4e4c-b52f-092d5a911104 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:05.940787107 +0000 UTC m=+35.132079595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls") pod "dns-default-47flm" (UID: "e6adbcf4-6061-4e4c-b52f-092d5a911104") : secret "dns-default-metrics-tls" not found Aug 04 09:41:05.441083 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.441053 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6adbcf4-6061-4e4c-b52f-092d5a911104-tmp-dir\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.441441 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.441384 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6adbcf4-6061-4e4c-b52f-092d5a911104-config-volume\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.445291 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.445272 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c66c8c6-437d-4958-b76b-544874765c73-cert\") pod \"ingress-canary-htz5h\" (UID: \"6c66c8c6-437d-4958-b76b-544874765c73\") " pod="openshift-ingress-canary/ingress-canary-htz5h" Aug 04 09:41:05.453683 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.453661 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vsrr\" (UniqueName: \"kubernetes.io/projected/e6adbcf4-6061-4e4c-b52f-092d5a911104-kube-api-access-2vsrr\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.454284 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.454250 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh2rr\" (UniqueName: \"kubernetes.io/projected/6c66c8c6-437d-4958-b76b-544874765c73-kube-api-access-vh2rr\") pod \"ingress-canary-htz5h\" (UID: \"6c66c8c6-437d-4958-b76b-544874765c73\") " pod="openshift-ingress-canary/ingress-canary-htz5h" Aug 04 09:41:05.495384 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.495359 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-htz5h" Aug 04 09:41:05.641696 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.641664 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-htz5h"] Aug 04 09:41:05.646548 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:05.646519 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c66c8c6_437d_4958_b76b_544874765c73.slice/crio-4e008fe04bf3141b44e5eb96d9b5e09813c821307b76d90bbc5145c48d14ce50 WatchSource:0}: Error finding container 4e008fe04bf3141b44e5eb96d9b5e09813c821307b76d90bbc5145c48d14ce50: Status 404 returned error can't find the container with id 4e008fe04bf3141b44e5eb96d9b5e09813c821307b76d90bbc5145c48d14ce50 Aug 04 09:41:05.943175 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:05.943138 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:05.943322 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:05.943304 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:05.943378 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:05.943371 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls podName:e6adbcf4-6061-4e4c-b52f-092d5a911104 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:06.943355317 +0000 UTC m=+36.134647796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls") pod "dns-default-47flm" (UID: "e6adbcf4-6061-4e4c-b52f-092d5a911104") : secret "dns-default-metrics-tls" not found Aug 04 09:41:06.332244 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:06.331997 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:41:06.332395 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:06.332023 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:41:06.334352 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:06.334327 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 09:41:06.334834 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:06.334817 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zpz8l\"" Aug 04 09:41:06.335474 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:06.334942 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gfsnh\"" Aug 04 09:41:06.335474 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:06.335014 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 09:41:06.335474 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:06.335015 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 09:41:06.461209 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:06.461163 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-htz5h" event={"ID":"6c66c8c6-437d-4958-b76b-544874765c73","Type":"ContainerStarted","Data":"4e008fe04bf3141b44e5eb96d9b5e09813c821307b76d90bbc5145c48d14ce50"} Aug 04 09:41:06.952420 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:06.952382 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:06.952611 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:06.952516 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:06.952611 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:06.952579 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls podName:e6adbcf4-6061-4e4c-b52f-092d5a911104 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:08.95256162 +0000 UTC m=+38.143854093 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls") pod "dns-default-47flm" (UID: "e6adbcf4-6061-4e4c-b52f-092d5a911104") : secret "dns-default-metrics-tls" not found Aug 04 09:41:07.365414 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.365348 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-b5jm2"] Aug 04 09:41:07.367183 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.367165 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.371873 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.371849 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Aug 04 09:41:07.372434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.372361 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 04 09:41:07.372434 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.372417 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 04 09:41:07.372720 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.372699 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Aug 04 09:41:07.372832 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.372749 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fctkf\"" Aug 04 09:41:07.378577 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.378559 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Aug 04 09:41:07.379721 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.379700 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-b5jm2"] Aug 04 09:41:07.456709 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.456683 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-trusted-ca-bundle\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.456851 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.456756 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-tmp\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.456851 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.456791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-service-ca-bundle\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.456851 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.456816 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9qdw\" (UniqueName: \"kubernetes.io/projected/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-kube-api-access-f9qdw\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.456851 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.456848 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-serving-cert\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.457068 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.456925 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-snapshots\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.557684 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.557656 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-trusted-ca-bundle\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.557807 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.557712 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-tmp\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.557807 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.557780 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-service-ca-bundle\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.557908 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.557835 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9qdw\" (UniqueName: \"kubernetes.io/projected/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-kube-api-access-f9qdw\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.557908 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.557876 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-serving-cert\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.558068 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.558048 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-tmp\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.558068 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.558047 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-snapshots\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.558494 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.558453 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-service-ca-bundle\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.558599 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.558539 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-snapshots\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.558643 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.558631 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-trusted-ca-bundle\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.561507 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.561486 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-serving-cert\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.592567 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.592549 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9qdw\" (UniqueName: \"kubernetes.io/projected/bbfc0fc9-940f-4c54-a66d-ac422afa0ceb-kube-api-access-f9qdw\") pod \"insights-operator-77696685b-b5jm2\" (UID: \"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb\") " pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.596198 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.596085 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh"] Aug 04 09:41:07.598834 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.598818 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:07.603121 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.603098 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Aug 04 09:41:07.603465 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.603350 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hd9b9\"" Aug 04 09:41:07.603716 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.603694 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Aug 04 09:41:07.604092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.604074 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-t2tlf"] Aug 04 09:41:07.604287 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.604265 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:41:07.606898 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.606859 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.609416 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.609397 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Aug 04 09:41:07.609611 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.609596 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:41:07.609919 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.609707 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Aug 04 09:41:07.609919 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.609750 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Aug 04 09:41:07.609919 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.609795 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-pdcm8\"" Aug 04 09:41:07.616612 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.616594 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh"] Aug 04 09:41:07.619624 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.619593 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Aug 04 09:41:07.620843 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.620825 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-t2tlf"] Aug 04 09:41:07.678110 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.678088 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-b5jm2" Aug 04 09:41:07.759431 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.759391 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-config\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.759539 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.759491 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-serving-cert\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.759539 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.759528 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-trusted-ca\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.759647 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.759591 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpkgf\" (UniqueName: \"kubernetes.io/projected/dbf67808-815a-43d1-be6a-632ae17df0be-kube-api-access-kpkgf\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:07.759690 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.759660 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:07.759747 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.759696 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlrsw\" (UniqueName: \"kubernetes.io/projected/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-kube-api-access-wlrsw\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.860921 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.860892 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-serving-cert\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.860921 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.860925 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-trusted-ca\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.861129 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.860981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpkgf\" (UniqueName: \"kubernetes.io/projected/dbf67808-815a-43d1-be6a-632ae17df0be-kube-api-access-kpkgf\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:07.861129 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.861031 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:07.861129 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:07.861108 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 09:41:07.861271 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:07.861184 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls podName:dbf67808-815a-43d1-be6a-632ae17df0be nodeName:}" failed. No retries permitted until 2026-08-04 09:41:08.361163725 +0000 UTC m=+37.552456203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-xt5nh" (UID: "dbf67808-815a-43d1-be6a-632ae17df0be") : secret "samples-operator-tls" not found Aug 04 09:41:07.861271 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.861204 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlrsw\" (UniqueName: \"kubernetes.io/projected/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-kube-api-access-wlrsw\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.861377 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.861272 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-config\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.861969 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.861942 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-config\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.862080 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.861966 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-trusted-ca\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.863790 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.863770 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-serving-cert\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.870946 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.870893 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlrsw\" (UniqueName: \"kubernetes.io/projected/0215bf3b-ca0e-454c-b48e-0ec33d6a8f23-kube-api-access-wlrsw\") pod \"console-operator-5ccd6b445f-t2tlf\" (UID: \"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23\") " pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:07.870946 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.870925 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpkgf\" (UniqueName: \"kubernetes.io/projected/dbf67808-815a-43d1-be6a-632ae17df0be-kube-api-access-kpkgf\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:07.917973 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:07.917950 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:08.364938 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:08.364902 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:08.365103 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:08.365066 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 09:41:08.365164 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:08.365153 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls podName:dbf67808-815a-43d1-be6a-632ae17df0be nodeName:}" failed. No retries permitted until 2026-08-04 09:41:09.365131347 +0000 UTC m=+38.556423821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-xt5nh" (UID: "dbf67808-815a-43d1-be6a-632ae17df0be") : secret "samples-operator-tls" not found Aug 04 09:41:08.892613 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:08.892450 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-t2tlf"] Aug 04 09:41:08.895575 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:08.895541 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0215bf3b_ca0e_454c_b48e_0ec33d6a8f23.slice/crio-a0890ff9ec77c6463ba79a664653ed9e55d537d8e7bf780c63400bedf5ef5fba WatchSource:0}: Error finding container a0890ff9ec77c6463ba79a664653ed9e55d537d8e7bf780c63400bedf5ef5fba: Status 404 returned error can't find the container with id a0890ff9ec77c6463ba79a664653ed9e55d537d8e7bf780c63400bedf5ef5fba Aug 04 09:41:08.898795 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:08.898774 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-b5jm2"] Aug 04 09:41:08.901618 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:08.901595 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbfc0fc9_940f_4c54_a66d_ac422afa0ceb.slice/crio-ff47021d95ef959c2e469621ea6807d1a844b111c2b33b9509817bc16830b821 WatchSource:0}: Error finding container ff47021d95ef959c2e469621ea6807d1a844b111c2b33b9509817bc16830b821: Status 404 returned error can't find the container with id ff47021d95ef959c2e469621ea6807d1a844b111c2b33b9509817bc16830b821 Aug 04 09:41:08.969956 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:08.969920 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:08.970113 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:08.970060 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:08.970153 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:08.970125 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls podName:e6adbcf4-6061-4e4c-b52f-092d5a911104 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:12.970110338 +0000 UTC m=+42.161402815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls") pod "dns-default-47flm" (UID: "e6adbcf4-6061-4e4c-b52f-092d5a911104") : secret "dns-default-metrics-tls" not found Aug 04 09:41:09.372692 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:09.372624 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:09.372852 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:09.372722 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 09:41:09.372852 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:09.372788 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls podName:dbf67808-815a-43d1-be6a-632ae17df0be nodeName:}" failed. No retries permitted until 2026-08-04 09:41:11.372775284 +0000 UTC m=+40.564067758 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-xt5nh" (UID: "dbf67808-815a-43d1-be6a-632ae17df0be") : secret "samples-operator-tls" not found Aug 04 09:41:09.470576 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:09.470546 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-htz5h" event={"ID":"6c66c8c6-437d-4958-b76b-544874765c73","Type":"ContainerStarted","Data":"3b5153f11fc4414214f2d038b4c5ef05f6b70d5338d816cd465d2fa7a0f17c46"} Aug 04 09:41:09.471624 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:09.471601 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" event={"ID":"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23","Type":"ContainerStarted","Data":"a0890ff9ec77c6463ba79a664653ed9e55d537d8e7bf780c63400bedf5ef5fba"} Aug 04 09:41:09.472515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:09.472492 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-b5jm2" event={"ID":"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb","Type":"ContainerStarted","Data":"ff47021d95ef959c2e469621ea6807d1a844b111c2b33b9509817bc16830b821"} Aug 04 09:41:09.475131 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:09.475107 2575 generic.go:356] "Generic (PLEG): container finished" podID="6309670e-2dff-4216-b018-1ed218e5da34" containerID="68541ed8651bcb2474b51c4cfd9929a2b72e9baf8feccd774285f632f6b17633" exitCode=0 Aug 04 09:41:09.475213 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:09.475136 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" event={"ID":"6309670e-2dff-4216-b018-1ed218e5da34","Type":"ContainerDied","Data":"68541ed8651bcb2474b51c4cfd9929a2b72e9baf8feccd774285f632f6b17633"} Aug 04 09:41:09.488634 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:09.488581 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-htz5h" podStartSLOduration=1.381078793 podStartE2EDuration="4.488557575s" totalImagesPullingTime="3.107478782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:09.488231748 +0000 UTC m=+38.679524245" watchObservedRunningTime="2026-08-04 09:41:09.488557575 +0000 UTC m=+38.679850114" Aug 04 09:41:10.483172 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:10.482552 2575 generic.go:356] "Generic (PLEG): container finished" podID="6309670e-2dff-4216-b018-1ed218e5da34" containerID="d8db180bd7a9afb187d0bc09d3e8e1dcd4d86242769c4df4120edf879f5a8ce1" exitCode=0 Aug 04 09:41:10.483172 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:10.482849 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" event={"ID":"6309670e-2dff-4216-b018-1ed218e5da34","Type":"ContainerDied","Data":"d8db180bd7a9afb187d0bc09d3e8e1dcd4d86242769c4df4120edf879f5a8ce1"} Aug 04 09:41:11.388446 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:11.388379 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:11.388592 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:11.388539 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 09:41:11.388659 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:11.388608 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls podName:dbf67808-815a-43d1-be6a-632ae17df0be nodeName:}" failed. No retries permitted until 2026-08-04 09:41:15.388589738 +0000 UTC m=+44.579882212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-xt5nh" (UID: "dbf67808-815a-43d1-be6a-632ae17df0be") : secret "samples-operator-tls" not found Aug 04 09:41:12.391439 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.391387 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc"] Aug 04 09:41:12.413656 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.413636 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc"] Aug 04 09:41:12.413760 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.413720 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" Aug 04 09:41:12.420090 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.420029 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Aug 04 09:41:12.420277 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.420260 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Aug 04 09:41:12.420316 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.420286 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6djwx\"" Aug 04 09:41:12.420751 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.420718 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Aug 04 09:41:12.422882 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.422864 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:41:12.488700 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.488675 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/0.log" Aug 04 09:41:12.488795 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.488714 2575 generic.go:356] "Generic (PLEG): container finished" podID="0215bf3b-ca0e-454c-b48e-0ec33d6a8f23" containerID="bc0da222217720ec333faef63883ec633ebce11d5d9178c92f075ba9174f67ee" exitCode=255 Aug 04 09:41:12.488795 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.488767 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" event={"ID":"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23","Type":"ContainerDied","Data":"bc0da222217720ec333faef63883ec633ebce11d5d9178c92f075ba9174f67ee"} Aug 04 09:41:12.488981 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.488968 2575 scope.go:117] "RemoveContainer" containerID="bc0da222217720ec333faef63883ec633ebce11d5d9178c92f075ba9174f67ee" Aug 04 09:41:12.490418 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.490397 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-b5jm2" event={"ID":"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb","Type":"ContainerStarted","Data":"00992235cf15be8f906bb0f2d083e5df88fc1113d5e01609203c06a3373c3ed1"} Aug 04 09:41:12.493789 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.493769 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" event={"ID":"6309670e-2dff-4216-b018-1ed218e5da34","Type":"ContainerStarted","Data":"132f460e3ea9d1b1bf4c3ecd16517940db8e498a4bb85aa38ab26b089c198a85"} Aug 04 09:41:12.541721 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.541682 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-b5jm2" podStartSLOduration=2.430564103 podStartE2EDuration="5.541667533s" totalImagesPullingTime="3.11110343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:12.536762063 +0000 UTC m=+41.728054558" watchObservedRunningTime="2026-08-04 09:41:12.541667533 +0000 UTC m=+41.732960028" Aug 04 09:41:12.568307 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.568265 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qmwhl" podStartSLOduration=10.177842925 podStartE2EDuration="41.568250616s" totalImagesPullingTime="31.390407691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:12.567143527 +0000 UTC m=+41.758436023" watchObservedRunningTime="2026-08-04 09:41:12.568250616 +0000 UTC m=+41.759543111" Aug 04 09:41:12.596850 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.596826 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4120302-68f5-4b8b-b2bf-fc38adebec10-config\") pod \"service-ca-operator-784f97bcf6-8hrzc\" (UID: \"f4120302-68f5-4b8b-b2bf-fc38adebec10\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" Aug 04 09:41:12.596952 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.596865 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4120302-68f5-4b8b-b2bf-fc38adebec10-serving-cert\") pod \"service-ca-operator-784f97bcf6-8hrzc\" (UID: \"f4120302-68f5-4b8b-b2bf-fc38adebec10\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" Aug 04 09:41:12.597041 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.597017 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4lps\" (UniqueName: \"kubernetes.io/projected/f4120302-68f5-4b8b-b2bf-fc38adebec10-kube-api-access-d4lps\") pod \"service-ca-operator-784f97bcf6-8hrzc\" (UID: \"f4120302-68f5-4b8b-b2bf-fc38adebec10\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" Aug 04 09:41:12.697970 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.697949 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4lps\" (UniqueName: \"kubernetes.io/projected/f4120302-68f5-4b8b-b2bf-fc38adebec10-kube-api-access-d4lps\") pod \"service-ca-operator-784f97bcf6-8hrzc\" (UID: \"f4120302-68f5-4b8b-b2bf-fc38adebec10\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" Aug 04 09:41:12.698061 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.698004 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4120302-68f5-4b8b-b2bf-fc38adebec10-config\") pod \"service-ca-operator-784f97bcf6-8hrzc\" (UID: \"f4120302-68f5-4b8b-b2bf-fc38adebec10\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" Aug 04 09:41:12.698061 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.698042 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4120302-68f5-4b8b-b2bf-fc38adebec10-serving-cert\") pod \"service-ca-operator-784f97bcf6-8hrzc\" (UID: \"f4120302-68f5-4b8b-b2bf-fc38adebec10\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" Aug 04 09:41:12.698387 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.698370 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4120302-68f5-4b8b-b2bf-fc38adebec10-config\") pod \"service-ca-operator-784f97bcf6-8hrzc\" (UID: \"f4120302-68f5-4b8b-b2bf-fc38adebec10\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" Aug 04 09:41:12.700754 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.700723 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4120302-68f5-4b8b-b2bf-fc38adebec10-serving-cert\") pod \"service-ca-operator-784f97bcf6-8hrzc\" (UID: \"f4120302-68f5-4b8b-b2bf-fc38adebec10\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" Aug 04 09:41:12.712654 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.712627 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4lps\" (UniqueName: \"kubernetes.io/projected/f4120302-68f5-4b8b-b2bf-fc38adebec10-kube-api-access-d4lps\") pod \"service-ca-operator-784f97bcf6-8hrzc\" (UID: \"f4120302-68f5-4b8b-b2bf-fc38adebec10\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" Aug 04 09:41:12.721556 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.721536 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" Aug 04 09:41:12.861287 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:12.861259 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc"] Aug 04 09:41:12.866127 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:12.866097 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4120302_68f5_4b8b_b2bf_fc38adebec10.slice/crio-9369a97d98601e667222a8956cac891f2c5c8daa4a58b66d8ff8e26c68b491d6 WatchSource:0}: Error finding container 9369a97d98601e667222a8956cac891f2c5c8daa4a58b66d8ff8e26c68b491d6: Status 404 returned error can't find the container with id 9369a97d98601e667222a8956cac891f2c5c8daa4a58b66d8ff8e26c68b491d6 Aug 04 09:41:13.000975 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.000952 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:13.001103 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:13.001086 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:13.001154 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:13.001144 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls podName:e6adbcf4-6061-4e4c-b52f-092d5a911104 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:21.001127999 +0000 UTC m=+50.192420507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls") pod "dns-default-47flm" (UID: "e6adbcf4-6061-4e4c-b52f-092d5a911104") : secret "dns-default-metrics-tls" not found Aug 04 09:41:13.235724 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.235693 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jgcrm"] Aug 04 09:41:13.262796 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.262693 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.273746 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.270207 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-54lnd\"" Aug 04 09:41:13.273746 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.270558 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 04 09:41:13.273746 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.270766 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 04 09:41:13.275533 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.275513 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jgcrm"] Aug 04 09:41:13.404024 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.404001 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.404311 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.404046 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-crio-socket\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.404311 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.404086 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.404311 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.404131 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-data-volume\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.404311 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.404182 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcfhc\" (UniqueName: \"kubernetes.io/projected/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-api-access-zcfhc\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.498419 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.498391 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/1.log" Aug 04 09:41:13.498838 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.498818 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/0.log" Aug 04 09:41:13.498946 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.498851 2575 generic.go:356] "Generic (PLEG): container finished" podID="0215bf3b-ca0e-454c-b48e-0ec33d6a8f23" containerID="a2d4bd67185ac4645f07a273a2fc84db4074c251ff48d2378cc98fde9ea4d5ec" exitCode=255 Aug 04 09:41:13.498946 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.498918 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" event={"ID":"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23","Type":"ContainerDied","Data":"a2d4bd67185ac4645f07a273a2fc84db4074c251ff48d2378cc98fde9ea4d5ec"} Aug 04 09:41:13.499038 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.498980 2575 scope.go:117] "RemoveContainer" containerID="bc0da222217720ec333faef63883ec633ebce11d5d9178c92f075ba9174f67ee" Aug 04 09:41:13.499320 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.499176 2575 scope.go:117] "RemoveContainer" containerID="a2d4bd67185ac4645f07a273a2fc84db4074c251ff48d2378cc98fde9ea4d5ec" Aug 04 09:41:13.499403 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:13.499376 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-t2tlf_openshift-console-operator(0215bf3b-ca0e-454c-b48e-0ec33d6a8f23)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" podUID="0215bf3b-ca0e-454c-b48e-0ec33d6a8f23" Aug 04 09:41:13.500073 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.500050 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" event={"ID":"f4120302-68f5-4b8b-b2bf-fc38adebec10","Type":"ContainerStarted","Data":"9369a97d98601e667222a8956cac891f2c5c8daa4a58b66d8ff8e26c68b491d6"} Aug 04 09:41:13.505278 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.505255 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-crio-socket\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.505394 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.505315 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.505450 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.505427 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-data-volume\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.505507 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.505460 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcfhc\" (UniqueName: \"kubernetes.io/projected/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-api-access-zcfhc\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.505507 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.505483 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.505600 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.505505 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-crio-socket\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.505600 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:13.505551 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:13.505600 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:13.505572 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 09:41:13.505759 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:13.505614 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls podName:d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:14.005596504 +0000 UTC m=+43.196888984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgcrm" (UID: "d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:13.505759 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:13.505634 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm podName:d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:14.005624284 +0000 UTC m=+43.196916757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgcrm" (UID: "d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c") : configmap "kube-rbac-proxy" not found Aug 04 09:41:13.505873 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.505769 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-data-volume\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.514304 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.514241 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcfhc\" (UniqueName: \"kubernetes.io/projected/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-api-access-zcfhc\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:13.617544 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.617511 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-6lfct"] Aug 04 09:41:13.648925 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.648902 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-6lfct"] Aug 04 09:41:13.649078 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.649014 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6lfct" Aug 04 09:41:13.651334 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.651304 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-82m9p\"" Aug 04 09:41:13.807318 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.807243 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7btv\" (UniqueName: \"kubernetes.io/projected/89392938-931b-4905-bf48-f74b6dbd8fb7-kube-api-access-c7btv\") pod \"network-check-source-65bdbbff45-6lfct\" (UID: \"89392938-931b-4905-bf48-f74b6dbd8fb7\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6lfct" Aug 04 09:41:13.908236 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.908201 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7btv\" (UniqueName: \"kubernetes.io/projected/89392938-931b-4905-bf48-f74b6dbd8fb7-kube-api-access-c7btv\") pod \"network-check-source-65bdbbff45-6lfct\" (UID: \"89392938-931b-4905-bf48-f74b6dbd8fb7\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6lfct" Aug 04 09:41:13.919138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.919112 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7btv\" (UniqueName: \"kubernetes.io/projected/89392938-931b-4905-bf48-f74b6dbd8fb7-kube-api-access-c7btv\") pod \"network-check-source-65bdbbff45-6lfct\" (UID: \"89392938-931b-4905-bf48-f74b6dbd8fb7\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6lfct" Aug 04 09:41:13.958758 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:13.958713 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6lfct" Aug 04 09:41:14.009696 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:14.009659 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:14.009878 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:14.009750 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:14.009999 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:14.009988 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 09:41:14.010141 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:14.010059 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm podName:d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:15.010038163 +0000 UTC m=+44.201330652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgcrm" (UID: "d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c") : configmap "kube-rbac-proxy" not found Aug 04 09:41:14.010141 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:14.010122 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:14.010252 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:14.010187 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls podName:d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:15.010170512 +0000 UTC m=+44.201462989 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgcrm" (UID: "d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:14.094466 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:14.094380 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-6lfct"] Aug 04 09:41:14.098968 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:14.098926 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89392938_931b_4905_bf48_f74b6dbd8fb7.slice/crio-df53549f260da6e6558852237df5bbabc2b26e81cd335f9b07b2a078e13316f0 WatchSource:0}: Error finding container df53549f260da6e6558852237df5bbabc2b26e81cd335f9b07b2a078e13316f0: Status 404 returned error can't find the container with id df53549f260da6e6558852237df5bbabc2b26e81cd335f9b07b2a078e13316f0 Aug 04 09:41:14.503796 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:14.503770 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/1.log" Aug 04 09:41:14.504177 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:14.504120 2575 scope.go:117] "RemoveContainer" containerID="a2d4bd67185ac4645f07a273a2fc84db4074c251ff48d2378cc98fde9ea4d5ec" Aug 04 09:41:14.504282 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:14.504262 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-t2tlf_openshift-console-operator(0215bf3b-ca0e-454c-b48e-0ec33d6a8f23)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" podUID="0215bf3b-ca0e-454c-b48e-0ec33d6a8f23" Aug 04 09:41:14.505014 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:14.504989 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6lfct" event={"ID":"89392938-931b-4905-bf48-f74b6dbd8fb7","Type":"ContainerStarted","Data":"df53549f260da6e6558852237df5bbabc2b26e81cd335f9b07b2a078e13316f0"} Aug 04 09:41:15.016210 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:15.016168 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:15.016400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:15.016229 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:15.016400 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:15.016341 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 09:41:15.016400 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:15.016401 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm podName:d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:17.016383465 +0000 UTC m=+46.207675941 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgcrm" (UID: "d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c") : configmap "kube-rbac-proxy" not found Aug 04 09:41:15.016798 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:15.016764 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:15.016908 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:15.016828 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls podName:d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:17.016810621 +0000 UTC m=+46.208103110 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgcrm" (UID: "d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:15.369097 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:15.369031 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vkqkd_8612b3d8-d52c-4b28-96b1-c9202698e69d/dns-node-resolver/0.log" Aug 04 09:41:15.418835 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:15.418803 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:15.418979 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:15.418962 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 09:41:15.419049 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:15.419038 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls podName:dbf67808-815a-43d1-be6a-632ae17df0be nodeName:}" failed. No retries permitted until 2026-08-04 09:41:23.419015914 +0000 UTC m=+52.610308387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-xt5nh" (UID: "dbf67808-815a-43d1-be6a-632ae17df0be") : secret "samples-operator-tls" not found Aug 04 09:41:16.169627 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:16.169597 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-slfll_b16b5696-7aaa-4c86-928c-09e2f6d8f8b4/node-ca/0.log" Aug 04 09:41:16.511216 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:16.511171 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" event={"ID":"f4120302-68f5-4b8b-b2bf-fc38adebec10","Type":"ContainerStarted","Data":"8147a414c7a7bffaa1bfe574d31545bd5d4b387076149652e9717cba4f2f0da1"} Aug 04 09:41:16.528295 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:16.528240 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" podStartSLOduration=1.910188826 podStartE2EDuration="4.52822358s" totalImagesPullingTime="2.618034754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:16.527050258 +0000 UTC m=+45.718342753" watchObservedRunningTime="2026-08-04 09:41:16.52822358 +0000 UTC m=+45.719516077" Aug 04 09:41:16.574027 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:16.573938 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-htz5h_6c66c8c6-437d-4958-b76b-544874765c73/serve-healthcheck-canary/0.log" Aug 04 09:41:17.032652 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.032608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:17.032859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.032674 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:17.032859 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:17.032799 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:17.032859 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:17.032813 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 09:41:17.033027 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:17.032876 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls podName:d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:21.032853724 +0000 UTC m=+50.224146217 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgcrm" (UID: "d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:17.033027 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:17.032895 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm podName:d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:21.032886438 +0000 UTC m=+50.224178911 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgcrm" (UID: "d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c") : configmap "kube-rbac-proxy" not found Aug 04 09:41:17.604551 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.604530 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-vq486"] Aug 04 09:41:17.625633 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.625611 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-vq486"] Aug 04 09:41:17.625718 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.625701 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:17.632050 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.632027 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Aug 04 09:41:17.632283 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.632265 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Aug 04 09:41:17.632474 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.632396 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dmrr8\"" Aug 04 09:41:17.636847 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.636826 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-vq486\" (UID: \"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:17.636941 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.636904 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-vq486\" (UID: \"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:17.737734 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.737706 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-vq486\" (UID: \"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:17.737867 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.737829 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-vq486\" (UID: \"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:17.737913 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:17.737874 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 09:41:17.737946 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:17.737935 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert podName:a00b5360-ca2b-4ed4-a9ad-1d5e1539116c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:18.237918104 +0000 UTC m=+47.429210581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-vq486" (UID: "a00b5360-ca2b-4ed4-a9ad-1d5e1539116c") : secret "networking-console-plugin-cert" not found Aug 04 09:41:17.738311 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.738295 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-vq486\" (UID: \"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:17.919127 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.919054 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:17.919242 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.919141 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:17.919466 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:17.919453 2575 scope.go:117] "RemoveContainer" containerID="a2d4bd67185ac4645f07a273a2fc84db4074c251ff48d2378cc98fde9ea4d5ec" Aug 04 09:41:17.919621 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:17.919603 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-t2tlf_openshift-console-operator(0215bf3b-ca0e-454c-b48e-0ec33d6a8f23)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" podUID="0215bf3b-ca0e-454c-b48e-0ec33d6a8f23" Aug 04 09:41:18.241762 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:18.241716 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-vq486\" (UID: \"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:18.241961 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:18.241835 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 09:41:18.241961 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:18.241903 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert podName:a00b5360-ca2b-4ed4-a9ad-1d5e1539116c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:19.241879572 +0000 UTC m=+48.433172067 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-vq486" (UID: "a00b5360-ca2b-4ed4-a9ad-1d5e1539116c") : secret "networking-console-plugin-cert" not found Aug 04 09:41:18.516635 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:18.516558 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6lfct" event={"ID":"89392938-931b-4905-bf48-f74b6dbd8fb7","Type":"ContainerStarted","Data":"c59e6c51e474ec83481de8d0139d192d6c9f8af5552ab54fd116ddb4f5b403c1"} Aug 04 09:41:18.532320 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:18.532272 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6lfct" podStartSLOduration=2.049243841 podStartE2EDuration="5.532256728s" totalImagesPullingTime="3.483012887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:18.53214389 +0000 UTC m=+47.723436386" watchObservedRunningTime="2026-08-04 09:41:18.532256728 +0000 UTC m=+47.723549225" Aug 04 09:41:19.249632 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:19.249598 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-vq486\" (UID: \"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:19.250017 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:19.249717 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 09:41:19.250017 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:19.249792 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert podName:a00b5360-ca2b-4ed4-a9ad-1d5e1539116c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:21.249779131 +0000 UTC m=+50.441071604 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-vq486" (UID: "a00b5360-ca2b-4ed4-a9ad-1d5e1539116c") : secret "networking-console-plugin-cert" not found Aug 04 09:41:21.061680 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:21.061649 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:21.062056 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:21.061690 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:21.062056 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:21.061715 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:21.062056 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:21.061800 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 09:41:21.062056 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:21.061827 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:21.062056 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:21.061872 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:21.062056 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:21.061882 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm podName:d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:29.061861751 +0000 UTC m=+58.253154226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgcrm" (UID: "d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c") : configmap "kube-rbac-proxy" not found Aug 04 09:41:21.062056 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:21.061902 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls podName:d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:29.061891814 +0000 UTC m=+58.253184290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgcrm" (UID: "d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:21.062056 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:21.061915 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls podName:e6adbcf4-6061-4e4c-b52f-092d5a911104 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:37.061908307 +0000 UTC m=+66.253200780 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls") pod "dns-default-47flm" (UID: "e6adbcf4-6061-4e4c-b52f-092d5a911104") : secret "dns-default-metrics-tls" not found Aug 04 09:41:21.262767 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:21.262724 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-vq486\" (UID: \"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:21.262912 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:21.262858 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 09:41:21.262912 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:21.262912 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert podName:a00b5360-ca2b-4ed4-a9ad-1d5e1539116c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:25.262898396 +0000 UTC m=+54.454190870 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-vq486" (UID: "a00b5360-ca2b-4ed4-a9ad-1d5e1539116c") : secret "networking-console-plugin-cert" not found Aug 04 09:41:23.481535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:23.481501 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:23.481915 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:23.481639 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 09:41:23.481915 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:23.481714 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls podName:dbf67808-815a-43d1-be6a-632ae17df0be nodeName:}" failed. No retries permitted until 2026-08-04 09:41:39.481697902 +0000 UTC m=+68.672990380 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-xt5nh" (UID: "dbf67808-815a-43d1-be6a-632ae17df0be") : secret "samples-operator-tls" not found Aug 04 09:41:25.297202 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:25.297167 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-vq486\" (UID: \"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:25.297597 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:25.297277 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 09:41:25.297597 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:25.297338 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert podName:a00b5360-ca2b-4ed4-a9ad-1d5e1539116c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:33.29731299 +0000 UTC m=+62.488605463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-vq486" (UID: "a00b5360-ca2b-4ed4-a9ad-1d5e1539116c") : secret "networking-console-plugin-cert" not found Aug 04 09:41:29.126053 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:29.126015 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:29.126053 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:29.126060 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:29.126570 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:29.126156 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:29.126570 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:29.126222 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls podName:d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c nodeName:}" failed. No retries permitted until 2026-08-04 09:41:45.12620687 +0000 UTC m=+74.317499343 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgcrm" (UID: "d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:29.126570 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:29.126504 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:30.455848 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:30.455822 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rb78b" Aug 04 09:41:32.332222 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:32.332193 2575 scope.go:117] "RemoveContainer" containerID="a2d4bd67185ac4645f07a273a2fc84db4074c251ff48d2378cc98fde9ea4d5ec" Aug 04 09:41:32.550622 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:32.550598 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 09:41:32.550990 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:32.550976 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/1.log" Aug 04 09:41:32.551039 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:32.551010 2575 generic.go:356] "Generic (PLEG): container finished" podID="0215bf3b-ca0e-454c-b48e-0ec33d6a8f23" containerID="732ad25325ad84933ac540914edd8269b6793421ffb3cacb9f052c02fac0801b" exitCode=255 Aug 04 09:41:32.551072 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:32.551051 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" event={"ID":"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23","Type":"ContainerDied","Data":"732ad25325ad84933ac540914edd8269b6793421ffb3cacb9f052c02fac0801b"} Aug 04 09:41:32.551111 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:32.551102 2575 scope.go:117] "RemoveContainer" containerID="a2d4bd67185ac4645f07a273a2fc84db4074c251ff48d2378cc98fde9ea4d5ec" Aug 04 09:41:32.551375 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:32.551359 2575 scope.go:117] "RemoveContainer" containerID="732ad25325ad84933ac540914edd8269b6793421ffb3cacb9f052c02fac0801b" Aug 04 09:41:32.551540 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:32.551522 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-t2tlf_openshift-console-operator(0215bf3b-ca0e-454c-b48e-0ec33d6a8f23)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" podUID="0215bf3b-ca0e-454c-b48e-0ec33d6a8f23" Aug 04 09:41:33.356131 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:33.356104 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-vq486\" (UID: \"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:33.358301 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:33.358283 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a00b5360-ca2b-4ed4-a9ad-1d5e1539116c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-vq486\" (UID: \"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:33.546028 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:33.546001 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dmrr8\"" Aug 04 09:41:33.553939 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:33.553920 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" Aug 04 09:41:33.555140 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:33.555121 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 09:41:33.685555 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:33.685524 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-vq486"] Aug 04 09:41:33.689959 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:33.689932 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00b5360_ca2b_4ed4_a9ad_1d5e1539116c.slice/crio-045c8cb5cb60c8e87fedeebb43d14522855db1b54cc4df5cba6e73296c182632 WatchSource:0}: Error finding container 045c8cb5cb60c8e87fedeebb43d14522855db1b54cc4df5cba6e73296c182632: Status 404 returned error can't find the container with id 045c8cb5cb60c8e87fedeebb43d14522855db1b54cc4df5cba6e73296c182632 Aug 04 09:41:34.558955 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:34.558920 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" event={"ID":"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c","Type":"ContainerStarted","Data":"045c8cb5cb60c8e87fedeebb43d14522855db1b54cc4df5cba6e73296c182632"} Aug 04 09:41:35.562881 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:35.562843 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" event={"ID":"a00b5360-ca2b-4ed4-a9ad-1d5e1539116c","Type":"ContainerStarted","Data":"98e057c88e79ab421dcfb841cc3a112b9a944b62572f8281e029769db394520c"} Aug 04 09:41:35.580797 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:35.580757 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-vq486" podStartSLOduration=17.502311285 podStartE2EDuration="18.580722362s" totalImagesPullingTime="1.078411077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:35.579976768 +0000 UTC m=+64.771269263" watchObservedRunningTime="2026-08-04 09:41:35.580722362 +0000 UTC m=+64.772014857" Aug 04 09:41:36.985139 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:36.985092 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:41:36.985139 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:36.985144 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:41:36.987478 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:36.987456 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 09:41:36.987906 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:36.987889 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 09:41:36.998406 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:36.998385 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f192c223-2f86-4ee1-836a-60af7de6824a-metrics-certs\") pod \"network-metrics-daemon-pcs94\" (UID: \"f192c223-2f86-4ee1-836a-60af7de6824a\") " pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:41:36.998481 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:36.998387 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2004042-19e9-46e9-8379-ad03ca29d486-original-pull-secret\") pod \"global-pull-secret-syncer-g6s9t\" (UID: \"a2004042-19e9-46e9-8379-ad03ca29d486\") " pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:41:37.086024 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.085986 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:37.086126 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.086061 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tj67\" (UniqueName: \"kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67\") pod \"network-check-target-c6bvn\" (UID: \"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7\") " pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:41:37.088024 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.087997 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6adbcf4-6061-4e4c-b52f-092d5a911104-metrics-tls\") pod \"dns-default-47flm\" (UID: \"e6adbcf4-6061-4e4c-b52f-092d5a911104\") " pod="openshift-dns/dns-default-47flm" Aug 04 09:41:37.088200 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.088181 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tj67\" (UniqueName: \"kubernetes.io/projected/9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7-kube-api-access-6tj67\") pod \"network-check-target-c6bvn\" (UID: \"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7\") " pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:41:37.142092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.142070 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6s9t" Aug 04 09:41:37.248089 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.248028 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zpz8l\"" Aug 04 09:41:37.253184 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.253163 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gfsnh\"" Aug 04 09:41:37.256782 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.256765 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pcs94" Aug 04 09:41:37.261775 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.261759 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:41:37.266561 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.266537 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g6s9t"] Aug 04 09:41:37.269895 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:37.269867 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2004042_19e9_46e9_8379_ad03ca29d486.slice/crio-0a66cc6839912e80e534a0b8e69820769aa88fcaccd8e9ee0aa182e3a930b0d2 WatchSource:0}: Error finding container 0a66cc6839912e80e534a0b8e69820769aa88fcaccd8e9ee0aa182e3a930b0d2: Status 404 returned error can't find the container with id 0a66cc6839912e80e534a0b8e69820769aa88fcaccd8e9ee0aa182e3a930b0d2 Aug 04 09:41:37.291756 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.291722 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-psh6m\"" Aug 04 09:41:37.299596 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.299573 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-47flm" Aug 04 09:41:37.391982 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.391906 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pcs94"] Aug 04 09:41:37.395094 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:37.395064 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf192c223_2f86_4ee1_836a_60af7de6824a.slice/crio-2c98c4180e4d3b8372607da3803feddba27556161246747f6f4d493401ac5bed WatchSource:0}: Error finding container 2c98c4180e4d3b8372607da3803feddba27556161246747f6f4d493401ac5bed: Status 404 returned error can't find the container with id 2c98c4180e4d3b8372607da3803feddba27556161246747f6f4d493401ac5bed Aug 04 09:41:37.414698 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.414676 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c6bvn"] Aug 04 09:41:37.418055 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:37.418028 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a9f372a_83cf_4dd4_b8d1_e90ea6086ff7.slice/crio-249a815a74c7fe227b06b9bc6edcccd55c02dbefad90fb7ffc9908483a41102a WatchSource:0}: Error finding container 249a815a74c7fe227b06b9bc6edcccd55c02dbefad90fb7ffc9908483a41102a: Status 404 returned error can't find the container with id 249a815a74c7fe227b06b9bc6edcccd55c02dbefad90fb7ffc9908483a41102a Aug 04 09:41:37.433537 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.433512 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-47flm"] Aug 04 09:41:37.436273 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:37.436251 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6adbcf4_6061_4e4c_b52f_092d5a911104.slice/crio-c7af3073d438d8d8172c6dc421e1c9582c041ad4ffe8c972891830f0a6b3261b WatchSource:0}: Error finding container c7af3073d438d8d8172c6dc421e1c9582c041ad4ffe8c972891830f0a6b3261b: Status 404 returned error can't find the container with id c7af3073d438d8d8172c6dc421e1c9582c041ad4ffe8c972891830f0a6b3261b Aug 04 09:41:37.569348 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.569294 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pcs94" event={"ID":"f192c223-2f86-4ee1-836a-60af7de6824a","Type":"ContainerStarted","Data":"2c98c4180e4d3b8372607da3803feddba27556161246747f6f4d493401ac5bed"} Aug 04 09:41:37.570348 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.570328 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g6s9t" event={"ID":"a2004042-19e9-46e9-8379-ad03ca29d486","Type":"ContainerStarted","Data":"0a66cc6839912e80e534a0b8e69820769aa88fcaccd8e9ee0aa182e3a930b0d2"} Aug 04 09:41:37.571374 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.571345 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-47flm" event={"ID":"e6adbcf4-6061-4e4c-b52f-092d5a911104","Type":"ContainerStarted","Data":"c7af3073d438d8d8172c6dc421e1c9582c041ad4ffe8c972891830f0a6b3261b"} Aug 04 09:41:37.572667 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.572648 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c6bvn" event={"ID":"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7","Type":"ContainerStarted","Data":"d8f4f27f222372d06608b33636280a631774c02aa5eb1ea43f600b13c357726d"} Aug 04 09:41:37.572777 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.572678 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c6bvn" event={"ID":"9a9f372a-83cf-4dd4-b8d1-e90ea6086ff7","Type":"ContainerStarted","Data":"249a815a74c7fe227b06b9bc6edcccd55c02dbefad90fb7ffc9908483a41102a"} Aug 04 09:41:37.572847 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.572782 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:41:37.591534 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.591498 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c6bvn" podStartSLOduration=66.591484799 podStartE2EDuration="1m6.591484799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:37.589932114 +0000 UTC m=+66.781224609" watchObservedRunningTime="2026-08-04 09:41:37.591484799 +0000 UTC m=+66.782777288" Aug 04 09:41:37.919856 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.918880 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:37.919856 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.918955 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:41:37.919856 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:37.919290 2575 scope.go:117] "RemoveContainer" containerID="732ad25325ad84933ac540914edd8269b6793421ffb3cacb9f052c02fac0801b" Aug 04 09:41:37.919856 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:37.919460 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-t2tlf_openshift-console-operator(0215bf3b-ca0e-454c-b48e-0ec33d6a8f23)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" podUID="0215bf3b-ca0e-454c-b48e-0ec33d6a8f23" Aug 04 09:41:38.579604 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:38.579259 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pcs94" event={"ID":"f192c223-2f86-4ee1-836a-60af7de6824a","Type":"ContainerStarted","Data":"abdd4d8b14c01b67c65372f6466703f18f1b93dad6438746b92f48755bb6ba55"} Aug 04 09:41:38.579604 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:38.579348 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pcs94" event={"ID":"f192c223-2f86-4ee1-836a-60af7de6824a","Type":"ContainerStarted","Data":"d6b1dc9e213ed82a3bd195ac00c89d6e8077eb8f9d4a32748bade6bcdd4cf44d"} Aug 04 09:41:38.597962 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:38.597900 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pcs94" podStartSLOduration=66.63946799 podStartE2EDuration="1m7.597865889s" totalImagesPullingTime="958.397899ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:38.597116284 +0000 UTC m=+67.788408783" watchObservedRunningTime="2026-08-04 09:41:38.597865889 +0000 UTC m=+67.789158385" Aug 04 09:41:39.505035 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:39.504990 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:39.507646 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:39.507615 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbf67808-815a-43d1-be6a-632ae17df0be-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xt5nh\" (UID: \"dbf67808-815a-43d1-be6a-632ae17df0be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:39.584667 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:39.584626 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-47flm" event={"ID":"e6adbcf4-6061-4e4c-b52f-092d5a911104","Type":"ContainerStarted","Data":"77e74641e4587cb6532d782696d8c13dc8200a72e1ebfd60ff1105e588d8aa18"} Aug 04 09:41:39.585075 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:39.584687 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-47flm" event={"ID":"e6adbcf4-6061-4e4c-b52f-092d5a911104","Type":"ContainerStarted","Data":"e843ff047b70b56b21c07260564094d49b5557cec13a189906dc7589cbbfabc9"} Aug 04 09:41:39.607341 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:39.607295 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-47flm" podStartSLOduration=32.873289682 podStartE2EDuration="34.607276546s" totalImagesPullingTime="1.733986864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:39.607139964 +0000 UTC m=+68.798432458" watchObservedRunningTime="2026-08-04 09:41:39.607276546 +0000 UTC m=+68.798569098" Aug 04 09:41:39.711393 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:39.711365 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hd9b9\"" Aug 04 09:41:39.719518 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:39.719497 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" Aug 04 09:41:40.589074 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:40.589038 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-47flm" Aug 04 09:41:41.136325 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:41.136299 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh"] Aug 04 09:41:41.593669 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:41.593629 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g6s9t" event={"ID":"a2004042-19e9-46e9-8379-ad03ca29d486","Type":"ContainerStarted","Data":"6f4c96703b0727278a271b6e9321875aff349a5f10db21f774d7483c2d9bde50"} Aug 04 09:41:41.594875 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:41.594849 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" event={"ID":"dbf67808-815a-43d1-be6a-632ae17df0be","Type":"ContainerStarted","Data":"992dbb11c9b0439c4cff7ab66e652e4cb3072ed2cde7c5a129e6cc4150da67ad"} Aug 04 09:41:41.611653 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:41.611609 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-g6s9t" podStartSLOduration=66.810097686 podStartE2EDuration="1m10.611596129s" totalImagesPullingTime="3.801498443s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:41.610240887 +0000 UTC m=+70.801533381" watchObservedRunningTime="2026-08-04 09:41:41.611596129 +0000 UTC m=+70.802888624" Aug 04 09:41:43.090632 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.090594 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-555d69bfd7-wzgpr"] Aug 04 09:41:43.112195 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.112173 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.114439 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.114414 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6"] Aug 04 09:41:43.115107 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.114925 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Aug 04 09:41:43.115107 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.114954 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Aug 04 09:41:43.115942 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.115922 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-b4dvw\"" Aug 04 09:41:43.116059 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.115996 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Aug 04 09:41:43.132408 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.132390 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-555d69bfd7-wzgpr"] Aug 04 09:41:43.132499 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.132477 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:43.135018 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.134995 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Aug 04 09:41:43.135119 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.135035 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Aug 04 09:41:43.135119 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.135058 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Aug 04 09:41:43.135914 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.135896 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6"] Aug 04 09:41:43.136562 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.136547 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Aug 04 09:41:43.136835 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.136817 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Aug 04 09:41:43.203991 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.203966 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-555d69bfd7-wzgpr"] Aug 04 09:41:43.204162 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:43.204144 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-wtlsx registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" podUID="d689c210-1c39-4be3-93fd-64a1d867df25" Aug 04 09:41:43.204650 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.204633 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g"] Aug 04 09:41:43.222927 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.222907 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g" Aug 04 09:41:43.225847 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.225829 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-qsdcq\"" Aug 04 09:41:43.226112 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.226085 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Aug 04 09:41:43.236233 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.236212 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-registry-certificates\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.236314 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.236253 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d689c210-1c39-4be3-93fd-64a1d867df25-ca-trust-extracted\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.236314 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.236278 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-image-registry-private-configuration\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.236314 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.236298 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-installation-pull-secrets\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.236445 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.236404 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmfl6\" (UniqueName: \"kubernetes.io/projected/12aaf2c4-21f2-487b-89b2-6788cfe38524-kube-api-access-pmfl6\") pod \"klusterlet-addon-workmgr-58c67bf694-4bzf6\" (UID: \"12aaf2c4-21f2-487b-89b2-6788cfe38524\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:43.236490 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.236452 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-bound-sa-token\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.236544 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.236520 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-trusted-ca\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.236589 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.236572 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/12aaf2c4-21f2-487b-89b2-6788cfe38524-klusterlet-config\") pod \"klusterlet-addon-workmgr-58c67bf694-4bzf6\" (UID: \"12aaf2c4-21f2-487b-89b2-6788cfe38524\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:43.236651 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.236597 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-registry-tls\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.236651 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.236624 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtlsx\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-kube-api-access-wtlsx\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.236651 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.236647 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12aaf2c4-21f2-487b-89b2-6788cfe38524-tmp\") pod \"klusterlet-addon-workmgr-58c67bf694-4bzf6\" (UID: \"12aaf2c4-21f2-487b-89b2-6788cfe38524\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:43.242355 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.242337 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g"] Aug 04 09:41:43.337715 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.337692 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-image-registry-private-configuration\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.337829 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.337746 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-installation-pull-secrets\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.337829 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.337787 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/30fe3119-97af-4955-b7b5-a6128ed3fa56-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-gh27g\" (UID: \"30fe3119-97af-4955-b7b5-a6128ed3fa56\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g" Aug 04 09:41:43.337829 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.337822 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmfl6\" (UniqueName: \"kubernetes.io/projected/12aaf2c4-21f2-487b-89b2-6788cfe38524-kube-api-access-pmfl6\") pod \"klusterlet-addon-workmgr-58c67bf694-4bzf6\" (UID: \"12aaf2c4-21f2-487b-89b2-6788cfe38524\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:43.337979 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.337861 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-bound-sa-token\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.337979 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.337917 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-trusted-ca\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.337979 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.337954 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/12aaf2c4-21f2-487b-89b2-6788cfe38524-klusterlet-config\") pod \"klusterlet-addon-workmgr-58c67bf694-4bzf6\" (UID: \"12aaf2c4-21f2-487b-89b2-6788cfe38524\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:43.338122 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.337980 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-registry-tls\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.338122 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.338007 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtlsx\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-kube-api-access-wtlsx\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.338122 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.338030 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12aaf2c4-21f2-487b-89b2-6788cfe38524-tmp\") pod \"klusterlet-addon-workmgr-58c67bf694-4bzf6\" (UID: \"12aaf2c4-21f2-487b-89b2-6788cfe38524\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:43.338122 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.338096 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-registry-certificates\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.338852 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.338578 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d689c210-1c39-4be3-93fd-64a1d867df25-ca-trust-extracted\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.338852 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.338747 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12aaf2c4-21f2-487b-89b2-6788cfe38524-tmp\") pod \"klusterlet-addon-workmgr-58c67bf694-4bzf6\" (UID: \"12aaf2c4-21f2-487b-89b2-6788cfe38524\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:43.339011 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.338900 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d689c210-1c39-4be3-93fd-64a1d867df25-ca-trust-extracted\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.339322 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.339295 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-trusted-ca\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.339484 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.339463 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-registry-certificates\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.340574 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.340550 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-image-registry-private-configuration\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.340717 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.340674 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-registry-tls\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.340831 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.340814 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-installation-pull-secrets\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.341065 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.341045 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/12aaf2c4-21f2-487b-89b2-6788cfe38524-klusterlet-config\") pod \"klusterlet-addon-workmgr-58c67bf694-4bzf6\" (UID: \"12aaf2c4-21f2-487b-89b2-6788cfe38524\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:43.361308 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.361281 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-bound-sa-token\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.363188 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.363141 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtlsx\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-kube-api-access-wtlsx\") pod \"image-registry-555d69bfd7-wzgpr\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.374787 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.374767 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmfl6\" (UniqueName: \"kubernetes.io/projected/12aaf2c4-21f2-487b-89b2-6788cfe38524-kube-api-access-pmfl6\") pod \"klusterlet-addon-workmgr-58c67bf694-4bzf6\" (UID: \"12aaf2c4-21f2-487b-89b2-6788cfe38524\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:43.439420 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.439389 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/30fe3119-97af-4955-b7b5-a6128ed3fa56-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-gh27g\" (UID: \"30fe3119-97af-4955-b7b5-a6128ed3fa56\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g" Aug 04 09:41:43.441400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.441382 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/30fe3119-97af-4955-b7b5-a6128ed3fa56-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-gh27g\" (UID: \"30fe3119-97af-4955-b7b5-a6128ed3fa56\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g" Aug 04 09:41:43.442246 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.442221 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:43.530717 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.530687 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g" Aug 04 09:41:43.606449 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.606424 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.606578 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.606468 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" event={"ID":"dbf67808-815a-43d1-be6a-632ae17df0be","Type":"ContainerStarted","Data":"6e33a14d1705afea059071346c76928ef5f50a8a8f5a1156e44dc0b3ca80e262"} Aug 04 09:41:43.606578 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.606545 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" event={"ID":"dbf67808-815a-43d1-be6a-632ae17df0be","Type":"ContainerStarted","Data":"03a4b0e8afcd3fec452b960c2473c19afa2fad1f2e30f831dc5c881b85417d6e"} Aug 04 09:41:43.633222 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.633136 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:43.634364 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.634342 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6"] Aug 04 09:41:43.636630 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:43.636601 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12aaf2c4_21f2_487b_89b2_6788cfe38524.slice/crio-878e1a23ed0e68c8ef9789ab104e0d326f25f3428d81b382b552f610b53561a0 WatchSource:0}: Error finding container 878e1a23ed0e68c8ef9789ab104e0d326f25f3428d81b382b552f610b53561a0: Status 404 returned error can't find the container with id 878e1a23ed0e68c8ef9789ab104e0d326f25f3428d81b382b552f610b53561a0 Aug 04 09:41:43.707943 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.707922 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g"] Aug 04 09:41:43.708123 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:43.708098 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30fe3119_97af_4955_b7b5_a6128ed3fa56.slice/crio-004804c29bdb838f25d7545ad74f42c821ffd9b89f7f598a45e3cd6ac5be2a45 WatchSource:0}: Error finding container 004804c29bdb838f25d7545ad74f42c821ffd9b89f7f598a45e3cd6ac5be2a45: Status 404 returned error can't find the container with id 004804c29bdb838f25d7545ad74f42c821ffd9b89f7f598a45e3cd6ac5be2a45 Aug 04 09:41:43.740071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.740021 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xt5nh" podStartSLOduration=34.782169105 podStartE2EDuration="36.740002866s" totalImagesPullingTime="1.957833761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:43.739091947 +0000 UTC m=+72.930384442" watchObservedRunningTime="2026-08-04 09:41:43.740002866 +0000 UTC m=+72.931295362" Aug 04 09:41:43.741867 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.741850 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-installation-pull-secrets\") pod \"d689c210-1c39-4be3-93fd-64a1d867df25\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " Aug 04 09:41:43.741933 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.741885 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d689c210-1c39-4be3-93fd-64a1d867df25-ca-trust-extracted\") pod \"d689c210-1c39-4be3-93fd-64a1d867df25\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " Aug 04 09:41:43.741933 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.741906 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-trusted-ca\") pod \"d689c210-1c39-4be3-93fd-64a1d867df25\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " Aug 04 09:41:43.742005 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.741941 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtlsx\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-kube-api-access-wtlsx\") pod \"d689c210-1c39-4be3-93fd-64a1d867df25\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " Aug 04 09:41:43.742071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.742052 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-registry-tls\") pod \"d689c210-1c39-4be3-93fd-64a1d867df25\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " Aug 04 09:41:43.742112 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.742100 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-image-registry-private-configuration\") pod \"d689c210-1c39-4be3-93fd-64a1d867df25\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " Aug 04 09:41:43.742164 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.742135 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-registry-certificates\") pod \"d689c210-1c39-4be3-93fd-64a1d867df25\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " Aug 04 09:41:43.742215 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.742152 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d689c210-1c39-4be3-93fd-64a1d867df25-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d689c210-1c39-4be3-93fd-64a1d867df25" (UID: "d689c210-1c39-4be3-93fd-64a1d867df25"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:41:43.742215 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.742160 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-bound-sa-token\") pod \"d689c210-1c39-4be3-93fd-64a1d867df25\" (UID: \"d689c210-1c39-4be3-93fd-64a1d867df25\") " Aug 04 09:41:43.742896 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.742319 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d689c210-1c39-4be3-93fd-64a1d867df25" (UID: "d689c210-1c39-4be3-93fd-64a1d867df25"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:41:43.742896 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.742438 2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d689c210-1c39-4be3-93fd-64a1d867df25-ca-trust-extracted\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:41:43.742896 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.742456 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-trusted-ca\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:41:43.742896 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.742674 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d689c210-1c39-4be3-93fd-64a1d867df25" (UID: "d689c210-1c39-4be3-93fd-64a1d867df25"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:41:43.744372 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.744346 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-kube-api-access-wtlsx" (OuterVolumeSpecName: "kube-api-access-wtlsx") pod "d689c210-1c39-4be3-93fd-64a1d867df25" (UID: "d689c210-1c39-4be3-93fd-64a1d867df25"). InnerVolumeSpecName "kube-api-access-wtlsx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:41:43.744462 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.744403 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d689c210-1c39-4be3-93fd-64a1d867df25" (UID: "d689c210-1c39-4be3-93fd-64a1d867df25"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:41:43.744462 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.744416 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d689c210-1c39-4be3-93fd-64a1d867df25" (UID: "d689c210-1c39-4be3-93fd-64a1d867df25"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:41:43.744549 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.744526 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d689c210-1c39-4be3-93fd-64a1d867df25" (UID: "d689c210-1c39-4be3-93fd-64a1d867df25"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:41:43.744549 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.744532 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d689c210-1c39-4be3-93fd-64a1d867df25" (UID: "d689c210-1c39-4be3-93fd-64a1d867df25"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:41:43.843650 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.843612 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtlsx\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-kube-api-access-wtlsx\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:41:43.843650 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.843640 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-registry-tls\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:41:43.843650 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.843649 2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-image-registry-private-configuration\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:41:43.843650 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.843658 2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d689c210-1c39-4be3-93fd-64a1d867df25-registry-certificates\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:41:43.843926 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.843666 2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d689c210-1c39-4be3-93fd-64a1d867df25-bound-sa-token\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:41:43.843926 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:43.843675 2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d689c210-1c39-4be3-93fd-64a1d867df25-installation-pull-secrets\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:41:44.614327 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:44.614256 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g" event={"ID":"30fe3119-97af-4955-b7b5-a6128ed3fa56","Type":"ContainerStarted","Data":"004804c29bdb838f25d7545ad74f42c821ffd9b89f7f598a45e3cd6ac5be2a45"} Aug 04 09:41:44.616237 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:44.616148 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555d69bfd7-wzgpr" Aug 04 09:41:44.616448 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:44.616419 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" event={"ID":"12aaf2c4-21f2-487b-89b2-6788cfe38524","Type":"ContainerStarted","Data":"878e1a23ed0e68c8ef9789ab104e0d326f25f3428d81b382b552f610b53561a0"} Aug 04 09:41:44.664042 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:44.664013 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-555d69bfd7-wzgpr"] Aug 04 09:41:44.667991 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:44.667964 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-555d69bfd7-wzgpr"] Aug 04 09:41:45.155541 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:45.155504 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:45.158030 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:45.157997 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgcrm\" (UID: \"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c\") " pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:45.337509 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:45.337468 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d689c210-1c39-4be3-93fd-64a1d867df25" path="/var/lib/kubelet/pods/d689c210-1c39-4be3-93fd-64a1d867df25/volumes" Aug 04 09:41:45.377819 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:45.377789 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-54lnd\"" Aug 04 09:41:45.385911 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:45.385877 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jgcrm" Aug 04 09:41:47.610973 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:47.610942 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jgcrm"] Aug 04 09:41:47.613777 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:47.613750 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09d40a3_a8bd_4a62_af4c_bbbf6d437b3c.slice/crio-1d7c1f8011ccafee50f927b4e7bcfeaff388c2f8b60ea536844ea0668e528de0 WatchSource:0}: Error finding container 1d7c1f8011ccafee50f927b4e7bcfeaff388c2f8b60ea536844ea0668e528de0: Status 404 returned error can't find the container with id 1d7c1f8011ccafee50f927b4e7bcfeaff388c2f8b60ea536844ea0668e528de0 Aug 04 09:41:47.627713 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:47.627687 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgcrm" event={"ID":"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c","Type":"ContainerStarted","Data":"1d7c1f8011ccafee50f927b4e7bcfeaff388c2f8b60ea536844ea0668e528de0"} Aug 04 09:41:47.628934 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:47.628913 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g" event={"ID":"30fe3119-97af-4955-b7b5-a6128ed3fa56","Type":"ContainerStarted","Data":"4561694319edd0ef8d7ba8799f23e7f764b7261dfc396cb9ad91f24f6f16a104"} Aug 04 09:41:47.649977 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:47.649948 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g" Aug 04 09:41:47.654991 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:47.654974 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g" Aug 04 09:41:47.674407 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:47.674361 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-gh27g" podStartSLOduration=0.906859264 podStartE2EDuration="4.674347874s" totalImagesPullingTime="3.76748861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:47.67394479 +0000 UTC m=+76.865237284" watchObservedRunningTime="2026-08-04 09:41:47.674347874 +0000 UTC m=+76.865640393" Aug 04 09:41:48.633623 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:48.633593 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" event={"ID":"12aaf2c4-21f2-487b-89b2-6788cfe38524","Type":"ContainerStarted","Data":"622722353f1de176d3e6ffe79d0b06aec40723826d859c269acebad37a276822"} Aug 04 09:41:48.634050 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:48.633856 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:48.635415 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:48.635389 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgcrm" event={"ID":"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c","Type":"ContainerStarted","Data":"b9ddddd7e3c94d1da9518e4b3996c86fec97ad65f5740f3d16321a986d9279d9"} Aug 04 09:41:48.635683 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:48.635668 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" Aug 04 09:41:48.669889 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:48.669848 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58c67bf694-4bzf6" podStartSLOduration=1.3000096110000001 podStartE2EDuration="5.669835008s" totalImagesPullingTime="4.369825397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:48.665034176 +0000 UTC m=+77.856326671" watchObservedRunningTime="2026-08-04 09:41:48.669835008 +0000 UTC m=+77.861127482" Aug 04 09:41:49.337909 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:49.337881 2575 scope.go:117] "RemoveContainer" containerID="732ad25325ad84933ac540914edd8269b6793421ffb3cacb9f052c02fac0801b" Aug 04 09:41:49.338129 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:49.338107 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-t2tlf_openshift-console-operator(0215bf3b-ca0e-454c-b48e-0ec33d6a8f23)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" podUID="0215bf3b-ca0e-454c-b48e-0ec33d6a8f23" Aug 04 09:41:49.640819 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:49.640741 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgcrm" event={"ID":"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c","Type":"ContainerStarted","Data":"c66093081a7d963fa57bbb8ad63c6ab63e722a421a7d28dbab31141ede4523a7"} Aug 04 09:41:50.597383 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:50.597356 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-47flm" Aug 04 09:41:50.645966 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:50.645930 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgcrm" event={"ID":"d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c","Type":"ContainerStarted","Data":"dd23fd2c7e24c1dcc9d86903c30cebcddf5c37374c95d2f48ba84ca8bfc8e0f0"} Aug 04 09:41:50.681120 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:50.678057 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jgcrm" podStartSLOduration=35.562734162 podStartE2EDuration="37.67803759s" totalImagesPullingTime="2.115303428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:41:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:50.676838091 +0000 UTC m=+79.868130586" watchObservedRunningTime="2026-08-04 09:41:50.67803759 +0000 UTC m=+79.869330086" Aug 04 09:41:53.060098 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.060068 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk"] Aug 04 09:41:53.063083 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.063065 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.066592 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.066568 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Aug 04 09:41:53.066684 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.066569 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 04 09:41:53.066782 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.066758 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 04 09:41:53.068331 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.068312 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 04 09:41:53.068436 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.068335 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Aug 04 09:41:53.068586 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.068572 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dw8bt\"" Aug 04 09:41:53.083072 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.083053 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk"] Aug 04 09:41:53.107742 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.107706 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h"] Aug 04 09:41:53.109784 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.109767 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-g9r95"] Aug 04 09:41:53.109888 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.109865 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.111858 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.111835 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Aug 04 09:41:53.111858 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.111846 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Aug 04 09:41:53.112010 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.111897 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.112132 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.112116 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mzlxn\"" Aug 04 09:41:53.112399 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.112386 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Aug 04 09:41:53.113802 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.113789 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 04 09:41:53.114070 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.114059 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 04 09:41:53.114384 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.114370 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5gwbm\"" Aug 04 09:41:53.114745 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.114713 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 04 09:41:53.125967 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.125948 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h"] Aug 04 09:41:53.217354 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217321 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a87a755-5520-43e8-a8ca-5d986051ce4a-metrics-client-ca\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.217354 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217352 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13d57be2-3e45-40c4-b4df-f866902a5276-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.217550 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217378 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/13d57be2-3e45-40c4-b4df-f866902a5276-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.217550 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217420 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13d57be2-3e45-40c4-b4df-f866902a5276-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.217550 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217449 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.217550 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217468 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-wtmp\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.217550 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217491 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x48c6\" (UniqueName: \"kubernetes.io/projected/13d57be2-3e45-40c4-b4df-f866902a5276-kube-api-access-x48c6\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.217760 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217559 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-tls\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.217760 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217589 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.217760 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a87a755-5520-43e8-a8ca-5d986051ce4a-sys\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.217760 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217626 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a87a755-5520-43e8-a8ca-5d986051ce4a-root\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.217760 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217643 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-textfile\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.217760 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217677 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.217760 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217694 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jt7q\" (UniqueName: \"kubernetes.io/projected/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-api-access-9jt7q\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.218071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217790 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.218071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217832 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.218071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217889 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.218071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217917 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.218071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.217953 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnb6d\" (UniqueName: \"kubernetes.io/projected/5a87a755-5520-43e8-a8ca-5d986051ce4a-kube-api-access-qnb6d\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.318295 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318226 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a87a755-5520-43e8-a8ca-5d986051ce4a-metrics-client-ca\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.318295 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318259 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13d57be2-3e45-40c4-b4df-f866902a5276-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.318295 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318284 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/13d57be2-3e45-40c4-b4df-f866902a5276-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.318517 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318398 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13d57be2-3e45-40c4-b4df-f866902a5276-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.318517 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318440 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.318517 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318466 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-wtmp\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.318517 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318492 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x48c6\" (UniqueName: \"kubernetes.io/projected/13d57be2-3e45-40c4-b4df-f866902a5276-kube-api-access-x48c6\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.318706 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318521 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-tls\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.318706 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318548 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.318706 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318579 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a87a755-5520-43e8-a8ca-5d986051ce4a-sys\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.318706 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318631 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a87a755-5520-43e8-a8ca-5d986051ce4a-sys\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.318920 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:53.318717 2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Aug 04 09:41:53.318920 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:53.318808 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-tls podName:5a87a755-5520-43e8-a8ca-5d986051ce4a nodeName:}" failed. No retries permitted until 2026-08-04 09:41:53.818788368 +0000 UTC m=+83.010080843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-tls") pod "node-exporter-g9r95" (UID: "5a87a755-5520-43e8-a8ca-5d986051ce4a") : secret "node-exporter-tls" not found Aug 04 09:41:53.318920 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318891 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a87a755-5520-43e8-a8ca-5d986051ce4a-metrics-client-ca\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.319071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318938 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a87a755-5520-43e8-a8ca-5d986051ce4a-root\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.319071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318949 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-wtmp\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.319071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318964 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-textfile\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.319071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318994 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.319071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.318996 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a87a755-5520-43e8-a8ca-5d986051ce4a-root\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.319071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.319003 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13d57be2-3e45-40c4-b4df-f866902a5276-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.319071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.319020 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jt7q\" (UniqueName: \"kubernetes.io/projected/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-api-access-9jt7q\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.319071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.319068 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.319459 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.319102 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.319459 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.319139 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.319459 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.319168 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.319459 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.319198 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnb6d\" (UniqueName: \"kubernetes.io/projected/5a87a755-5520-43e8-a8ca-5d986051ce4a-kube-api-access-qnb6d\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.319459 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.319340 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.319459 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.319444 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.319761 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.319573 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-textfile\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.319761 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:53.319605 2575 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Aug 04 09:41:53.319761 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:53.319669 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-tls podName:6c4a6b2c-b806-4af1-aeeb-375b73f5519e nodeName:}" failed. No retries permitted until 2026-08-04 09:41:53.819654886 +0000 UTC m=+83.010947367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-tls") pod "kube-state-metrics-6bd9664c4-tmk4h" (UID: "6c4a6b2c-b806-4af1-aeeb-375b73f5519e") : secret "kube-state-metrics-tls" not found Aug 04 09:41:53.319919 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.319867 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.320032 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.320012 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.320960 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.320939 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/13d57be2-3e45-40c4-b4df-f866902a5276-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.321138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.321115 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.321209 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.321125 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13d57be2-3e45-40c4-b4df-f866902a5276-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.321625 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.321607 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.335774 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.335753 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnb6d\" (UniqueName: \"kubernetes.io/projected/5a87a755-5520-43e8-a8ca-5d986051ce4a-kube-api-access-qnb6d\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.338426 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.338404 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jt7q\" (UniqueName: \"kubernetes.io/projected/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-api-access-9jt7q\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.338565 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.338547 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x48c6\" (UniqueName: \"kubernetes.io/projected/13d57be2-3e45-40c4-b4df-f866902a5276-kube-api-access-x48c6\") pod \"openshift-state-metrics-6bb4d7b4dd-rsdmk\" (UID: \"13d57be2-3e45-40c4-b4df-f866902a5276\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.372544 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.372523 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" Aug 04 09:41:53.504723 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.504578 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk"] Aug 04 09:41:53.507168 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:53.507134 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d57be2_3e45_40c4_b4df_f866902a5276.slice/crio-c465535ddab48fed15cbdd653b1f8ca5d0899c9dab4213d673b5c28b25637ea2 WatchSource:0}: Error finding container c465535ddab48fed15cbdd653b1f8ca5d0899c9dab4213d673b5c28b25637ea2: Status 404 returned error can't find the container with id c465535ddab48fed15cbdd653b1f8ca5d0899c9dab4213d673b5c28b25637ea2 Aug 04 09:41:53.656719 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.656685 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" event={"ID":"13d57be2-3e45-40c4-b4df-f866902a5276","Type":"ContainerStarted","Data":"8a540172b0a1b9c2cd9b5eaa7d5b27cacdf7c9a84654129cba79bd7ada4c3ce3"} Aug 04 09:41:53.656878 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.656764 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" event={"ID":"13d57be2-3e45-40c4-b4df-f866902a5276","Type":"ContainerStarted","Data":"a400fa5042b8ec533e5442e6bb22121fb83cc4acb75c3a3f55f4eed69c348b34"} Aug 04 09:41:53.656878 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.656784 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" event={"ID":"13d57be2-3e45-40c4-b4df-f866902a5276","Type":"ContainerStarted","Data":"c465535ddab48fed15cbdd653b1f8ca5d0899c9dab4213d673b5c28b25637ea2"} Aug 04 09:41:53.823924 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.823892 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:53.824043 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.823954 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-tls\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.826119 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.826094 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a87a755-5520-43e8-a8ca-5d986051ce4a-node-exporter-tls\") pod \"node-exporter-g9r95\" (UID: \"5a87a755-5520-43e8-a8ca-5d986051ce4a\") " pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:53.826224 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:53.826132 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c4a6b2c-b806-4af1-aeeb-375b73f5519e-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-tmk4h\" (UID: \"6c4a6b2c-b806-4af1-aeeb-375b73f5519e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:54.019524 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:54.019493 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" Aug 04 09:41:54.024392 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:54.024369 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g9r95" Aug 04 09:41:54.034034 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:54.034011 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a87a755_5520_43e8_a8ca_5d986051ce4a.slice/crio-c1148a02c6dbffa19b7658df57c17c6e8660094a7698da35e9298e62767864a4 WatchSource:0}: Error finding container c1148a02c6dbffa19b7658df57c17c6e8660094a7698da35e9298e62767864a4: Status 404 returned error can't find the container with id c1148a02c6dbffa19b7658df57c17c6e8660094a7698da35e9298e62767864a4 Aug 04 09:41:54.157211 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:54.157085 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h"] Aug 04 09:41:54.159946 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:54.159915 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4a6b2c_b806_4af1_aeeb_375b73f5519e.slice/crio-88bed006ea94d61507ebbc817e3b4f6140072a327bb62710516b5b1ce1d5c412 WatchSource:0}: Error finding container 88bed006ea94d61507ebbc817e3b4f6140072a327bb62710516b5b1ce1d5c412: Status 404 returned error can't find the container with id 88bed006ea94d61507ebbc817e3b4f6140072a327bb62710516b5b1ce1d5c412 Aug 04 09:41:54.663415 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:54.663376 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" event={"ID":"13d57be2-3e45-40c4-b4df-f866902a5276","Type":"ContainerStarted","Data":"2593cd46afd49988302adf5ebf20ba42509046dc12be6ed242f43c96fcdd07f6"} Aug 04 09:41:54.664881 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:54.664838 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g9r95" event={"ID":"5a87a755-5520-43e8-a8ca-5d986051ce4a","Type":"ContainerStarted","Data":"c1148a02c6dbffa19b7658df57c17c6e8660094a7698da35e9298e62767864a4"} Aug 04 09:41:54.666058 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:54.666033 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" event={"ID":"6c4a6b2c-b806-4af1-aeeb-375b73f5519e","Type":"ContainerStarted","Data":"88bed006ea94d61507ebbc817e3b4f6140072a327bb62710516b5b1ce1d5c412"} Aug 04 09:41:54.686668 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:54.686618 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-rsdmk" podStartSLOduration=0.779505795 podStartE2EDuration="1.686603541s" totalImagesPullingTime="907.097746ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:54.685062206 +0000 UTC m=+83.876354680" watchObservedRunningTime="2026-08-04 09:41:54.686603541 +0000 UTC m=+83.877896035" Aug 04 09:41:55.012243 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.012208 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6964db96b7-vfn66"] Aug 04 09:41:55.016183 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.016156 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.110760 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.110717 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Aug 04 09:41:55.111270 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.111246 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Aug 04 09:41:55.111367 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.111285 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Aug 04 09:41:55.126792 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.126765 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Aug 04 09:41:55.127713 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.127692 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-497mb7q4aers5\"" Aug 04 09:41:55.128504 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.128481 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Aug 04 09:41:55.128904 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.128888 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-sprml\"" Aug 04 09:41:55.133628 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.133603 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.134121 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.133651 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm68t\" (UniqueName: \"kubernetes.io/projected/12726353-01f7-4908-940f-7da5a75bf5c0-kube-api-access-bm68t\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.134121 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.133715 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.134121 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.133787 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-tls\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.134121 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.133823 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12726353-01f7-4908-940f-7da5a75bf5c0-metrics-client-ca\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.134121 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.133891 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.134121 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.133917 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.134121 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.133942 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-grpc-tls\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.135562 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.135539 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6964db96b7-vfn66"] Aug 04 09:41:55.235329 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.235297 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.235783 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.235341 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.235783 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.235363 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-grpc-tls\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.235783 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.235412 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.235783 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.235457 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm68t\" (UniqueName: \"kubernetes.io/projected/12726353-01f7-4908-940f-7da5a75bf5c0-kube-api-access-bm68t\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.235783 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.235501 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.235783 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.235530 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-tls\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.235783 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.235562 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12726353-01f7-4908-940f-7da5a75bf5c0-metrics-client-ca\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.237117 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.236265 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12726353-01f7-4908-940f-7da5a75bf5c0-metrics-client-ca\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.238480 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.238316 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.238583 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.238548 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-grpc-tls\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.238676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.238652 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.245673 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.245564 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.247214 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.247192 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.248269 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.248248 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/12726353-01f7-4908-940f-7da5a75bf5c0-secret-thanos-querier-tls\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.258543 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.258524 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm68t\" (UniqueName: \"kubernetes.io/projected/12726353-01f7-4908-940f-7da5a75bf5c0-kube-api-access-bm68t\") pod \"thanos-querier-6964db96b7-vfn66\" (UID: \"12726353-01f7-4908-940f-7da5a75bf5c0\") " pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.327859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.327791 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:55.595611 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.594877 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6964db96b7-vfn66"] Aug 04 09:41:55.601112 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:55.601074 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12726353_01f7_4908_940f_7da5a75bf5c0.slice/crio-33c79a9bffce60d3b29f1df4431ba3cd7eef9afdf5ae5da58ce99050b74b72fc WatchSource:0}: Error finding container 33c79a9bffce60d3b29f1df4431ba3cd7eef9afdf5ae5da58ce99050b74b72fc: Status 404 returned error can't find the container with id 33c79a9bffce60d3b29f1df4431ba3cd7eef9afdf5ae5da58ce99050b74b72fc Aug 04 09:41:55.673296 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.673268 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" event={"ID":"12726353-01f7-4908-940f-7da5a75bf5c0","Type":"ContainerStarted","Data":"33c79a9bffce60d3b29f1df4431ba3cd7eef9afdf5ae5da58ce99050b74b72fc"} Aug 04 09:41:55.675129 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.675101 2575 generic.go:356] "Generic (PLEG): container finished" podID="5a87a755-5520-43e8-a8ca-5d986051ce4a" containerID="c9ea38117ae22748dd35d8e3c3c2590e0ef941d4b19222446911e38dc5b226f7" exitCode=0 Aug 04 09:41:55.675248 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.675216 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g9r95" event={"ID":"5a87a755-5520-43e8-a8ca-5d986051ce4a","Type":"ContainerDied","Data":"c9ea38117ae22748dd35d8e3c3c2590e0ef941d4b19222446911e38dc5b226f7"} Aug 04 09:41:55.677857 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.677830 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" event={"ID":"6c4a6b2c-b806-4af1-aeeb-375b73f5519e","Type":"ContainerStarted","Data":"f0c2c1fa51ec2c290f33c92fc87216cf07f21be40655b702985356ea5ebfc0c8"} Aug 04 09:41:55.677936 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.677890 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" event={"ID":"6c4a6b2c-b806-4af1-aeeb-375b73f5519e","Type":"ContainerStarted","Data":"a12a84b90d35e6af326c35a3f4e66a855576fb9ae868910c2d378986fc57bce6"} Aug 04 09:41:55.723814 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:55.723758 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" podStartSLOduration=1.405187573 podStartE2EDuration="2.723717176s" totalImagesPullingTime="1.318529603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:55.723517363 +0000 UTC m=+84.914809879" watchObservedRunningTime="2026-08-04 09:41:55.723717176 +0000 UTC m=+84.915009684" Aug 04 09:41:56.684419 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:56.684371 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-tmk4h" event={"ID":"6c4a6b2c-b806-4af1-aeeb-375b73f5519e","Type":"ContainerStarted","Data":"ed874f1147e416b45f3de5bfdb96e235c87636904c52315e8a015d3fae62c3f6"} Aug 04 09:41:56.686885 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:56.686858 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g9r95" event={"ID":"5a87a755-5520-43e8-a8ca-5d986051ce4a","Type":"ContainerStarted","Data":"933a081c6a7d498d0ba558d47314c10d35e3969f8c2d6bd0478f7b01b085faf6"} Aug 04 09:41:56.687021 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:56.686900 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g9r95" event={"ID":"5a87a755-5520-43e8-a8ca-5d986051ce4a","Type":"ContainerStarted","Data":"49664af21a83c8b66dae5e0be0b0d5d4e8554f4b55443132cab1c97ef81bdc51"} Aug 04 09:41:56.720532 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:56.720483 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-g9r95" podStartSLOduration=2.577774822 podStartE2EDuration="3.720468678s" totalImagesPullingTime="1.071448159s" totalInitContainerRuntime="71.245697ms" isStatefulPod=true podCreationTimestamp="2026-08-04 09:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:56.720254675 +0000 UTC m=+85.911547174" watchObservedRunningTime="2026-08-04 09:41:56.720468678 +0000 UTC m=+85.911761154" Aug 04 09:41:57.581378 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:57.581345 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf"] Aug 04 09:41:57.583997 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:57.583976 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" Aug 04 09:41:57.593415 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:57.593393 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Aug 04 09:41:57.594435 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:57.594416 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2x9vq\"" Aug 04 09:41:57.612859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:57.612828 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf"] Aug 04 09:41:57.759646 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:57.759607 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97f72632-fe67-4f4c-886c-52ccb66ad0e1-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-wf7mf\" (UID: \"97f72632-fe67-4f4c-886c-52ccb66ad0e1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" Aug 04 09:41:57.760091 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:57.759776 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdlcn\" (UniqueName: \"kubernetes.io/projected/97f72632-fe67-4f4c-886c-52ccb66ad0e1-kube-api-access-pdlcn\") pod \"monitoring-plugin-6dffbbd67-wf7mf\" (UID: \"97f72632-fe67-4f4c-886c-52ccb66ad0e1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" Aug 04 09:41:57.860599 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:57.860557 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdlcn\" (UniqueName: \"kubernetes.io/projected/97f72632-fe67-4f4c-886c-52ccb66ad0e1-kube-api-access-pdlcn\") pod \"monitoring-plugin-6dffbbd67-wf7mf\" (UID: \"97f72632-fe67-4f4c-886c-52ccb66ad0e1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" Aug 04 09:41:57.861068 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:57.860800 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97f72632-fe67-4f4c-886c-52ccb66ad0e1-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-wf7mf\" (UID: \"97f72632-fe67-4f4c-886c-52ccb66ad0e1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" Aug 04 09:41:57.861068 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:57.860868 2575 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Aug 04 09:41:57.861068 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:41:57.860974 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f72632-fe67-4f4c-886c-52ccb66ad0e1-monitoring-plugin-cert podName:97f72632-fe67-4f4c-886c-52ccb66ad0e1 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:58.360944833 +0000 UTC m=+87.552237322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/97f72632-fe67-4f4c-886c-52ccb66ad0e1-monitoring-plugin-cert") pod "monitoring-plugin-6dffbbd67-wf7mf" (UID: "97f72632-fe67-4f4c-886c-52ccb66ad0e1") : secret "monitoring-plugin-cert" not found Aug 04 09:41:57.869452 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:57.869429 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdlcn\" (UniqueName: \"kubernetes.io/projected/97f72632-fe67-4f4c-886c-52ccb66ad0e1-kube-api-access-pdlcn\") pod \"monitoring-plugin-6dffbbd67-wf7mf\" (UID: \"97f72632-fe67-4f4c-886c-52ccb66ad0e1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" Aug 04 09:41:58.365613 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:58.365579 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97f72632-fe67-4f4c-886c-52ccb66ad0e1-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-wf7mf\" (UID: \"97f72632-fe67-4f4c-886c-52ccb66ad0e1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" Aug 04 09:41:58.367838 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:58.367815 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97f72632-fe67-4f4c-886c-52ccb66ad0e1-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-wf7mf\" (UID: \"97f72632-fe67-4f4c-886c-52ccb66ad0e1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" Aug 04 09:41:58.495401 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:58.495372 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" Aug 04 09:41:58.663498 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:58.663413 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf"] Aug 04 09:41:58.668931 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:58.668895 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97f72632_fe67_4f4c_886c_52ccb66ad0e1.slice/crio-322c1e334df5cf8f10b90c6ae2e9f373f24a4965347a57a107c84e0af7680ef7 WatchSource:0}: Error finding container 322c1e334df5cf8f10b90c6ae2e9f373f24a4965347a57a107c84e0af7680ef7: Status 404 returned error can't find the container with id 322c1e334df5cf8f10b90c6ae2e9f373f24a4965347a57a107c84e0af7680ef7 Aug 04 09:41:58.696936 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:58.696882 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" event={"ID":"97f72632-fe67-4f4c-886c-52ccb66ad0e1","Type":"ContainerStarted","Data":"322c1e334df5cf8f10b90c6ae2e9f373f24a4965347a57a107c84e0af7680ef7"} Aug 04 09:41:58.699299 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:58.699276 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" event={"ID":"12726353-01f7-4908-940f-7da5a75bf5c0","Type":"ContainerStarted","Data":"8e99c9e8e064397f89defe926b95ca11dc2d493a9509169067a0c4f66b9518ae"} Aug 04 09:41:58.699406 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:58.699330 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" event={"ID":"12726353-01f7-4908-940f-7da5a75bf5c0","Type":"ContainerStarted","Data":"9650c624cca01fcb8e7f103085a5071a709ea7a6dfe20ed8423f2156742f2c0b"} Aug 04 09:41:58.699406 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:58.699356 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" event={"ID":"12726353-01f7-4908-940f-7da5a75bf5c0","Type":"ContainerStarted","Data":"c39e86bdedf60bb63b80880122d3dac6762d8b5a9c426b76b20141b117517d7a"} Aug 04 09:41:59.447918 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.447886 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 09:41:59.451960 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.451938 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.462824 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.462805 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Aug 04 09:41:59.466204 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.466179 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Aug 04 09:41:59.466421 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.466406 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Aug 04 09:41:59.466648 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.466629 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Aug 04 09:41:59.466893 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.466874 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-w88rc\"" Aug 04 09:41:59.467047 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.467029 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Aug 04 09:41:59.467153 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.467128 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Aug 04 09:41:59.467247 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.467138 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Aug 04 09:41:59.467743 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.467709 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Aug 04 09:41:59.467962 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.467941 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Aug 04 09:41:59.485029 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.485007 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Aug 04 09:41:59.485134 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.485028 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a0ofjc2sesvat\"" Aug 04 09:41:59.485278 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.485259 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Aug 04 09:41:59.485365 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.485311 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Aug 04 09:41:59.485502 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.485485 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Aug 04 09:41:59.486601 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.486587 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Aug 04 09:41:59.499276 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.499255 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Aug 04 09:41:59.517114 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.517088 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 09:41:59.576555 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.576528 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.576709 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.576561 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.576709 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.576589 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.576709 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.576611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jxc2\" (UniqueName: \"kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-kube-api-access-8jxc2\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.576878 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.576715 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.576878 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.576782 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.576878 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.576822 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.576878 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.576852 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577067 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.576923 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577067 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.576961 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577067 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.576996 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577067 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.577023 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577067 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.577062 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577266 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.577124 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577266 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.577147 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-web-config\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577266 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.577182 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config-out\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577266 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.577206 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577266 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.577229 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577438 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.577294 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.577438 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.577324 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.678601 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678571 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config-out\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.678601 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678606 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.678859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678630 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.678859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678662 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.678859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678685 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.678859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.678859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678748 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.678859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678775 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.678859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678796 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jxc2\" (UniqueName: \"kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-kube-api-access-8jxc2\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.678859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678837 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.679229 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678870 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.679229 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678907 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.679229 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678937 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.679229 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.678973 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.679229 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.679001 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.679229 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.679033 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.679229 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.679054 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.679229 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.679095 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.679229 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.679136 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.679229 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.679161 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-web-config\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.680538 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.679861 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.680538 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.680097 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.680710 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.680627 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.681016 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.680979 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.681562 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.681536 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.681832 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.681808 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.683009 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.682979 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.683194 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.683169 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.683421 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.683397 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-web-config\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.683505 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.683400 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config-out\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.683674 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.683649 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.684119 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.684098 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.684382 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.684357 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.684459 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.684432 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.684896 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.684875 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.685790 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.685747 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.686171 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.686131 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.686303 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.686279 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.687163 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.687141 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.688277 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.688253 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jxc2\" (UniqueName: \"kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-kube-api-access-8jxc2\") pod \"prometheus-k8s-0\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.708301 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.708234 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" event={"ID":"12726353-01f7-4908-940f-7da5a75bf5c0","Type":"ContainerStarted","Data":"3a71e80dcf0cc2d3beca7f637a05e60a51c1bda5c6afa1c8d12c70e312f9e853"} Aug 04 09:41:59.708390 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.708310 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" event={"ID":"12726353-01f7-4908-940f-7da5a75bf5c0","Type":"ContainerStarted","Data":"c22588d7b0e7fdc6defcd1f1e60751955c7468f450a33f6c905d4db45ff85201"} Aug 04 09:41:59.708390 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.708339 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" event={"ID":"12726353-01f7-4908-940f-7da5a75bf5c0","Type":"ContainerStarted","Data":"d6ff7a03cf5e3ba06aaa9278be425ff5477f5a9dc2344db2cbe3272b1f5a9b9e"} Aug 04 09:41:59.708484 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.708403 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:41:59.741939 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.741884 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" podStartSLOduration=2.670645777 podStartE2EDuration="5.741864737s" totalImagesPullingTime="3.07121896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:59.740240923 +0000 UTC m=+88.931533417" watchObservedRunningTime="2026-08-04 09:41:59.741864737 +0000 UTC m=+88.933157234" Aug 04 09:41:59.762445 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.762378 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:41:59.955513 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:41:59.955485 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 09:41:59.958245 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:41:59.958223 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4102bfb7_9ff3_41d4_af13_5a5f9a45d0c8.slice/crio-809544c1a512295e861052aa87cf0cde6b2dbbf595832d7559948c65015bedcc WatchSource:0}: Error finding container 809544c1a512295e861052aa87cf0cde6b2dbbf595832d7559948c65015bedcc: Status 404 returned error can't find the container with id 809544c1a512295e861052aa87cf0cde6b2dbbf595832d7559948c65015bedcc Aug 04 09:42:00.332644 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:00.332564 2575 scope.go:117] "RemoveContainer" containerID="732ad25325ad84933ac540914edd8269b6793421ffb3cacb9f052c02fac0801b" Aug 04 09:42:00.714041 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:00.714005 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" event={"ID":"97f72632-fe67-4f4c-886c-52ccb66ad0e1","Type":"ContainerStarted","Data":"fc201111166ee6048c423b3499e865dcb9c3fcbc1be60062a87352a4825b45d1"} Aug 04 09:42:00.714487 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:00.714068 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" Aug 04 09:42:00.716117 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:00.716085 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 09:42:00.716213 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:00.716177 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" event={"ID":"0215bf3b-ca0e-454c-b48e-0ec33d6a8f23","Type":"ContainerStarted","Data":"c833b50db7ec20dc3025159a291d449d6cf7bd80140e21e11374d21d7d763578"} Aug 04 09:42:00.716531 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:00.716478 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:42:00.717963 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:00.717935 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerStarted","Data":"809544c1a512295e861052aa87cf0cde6b2dbbf595832d7559948c65015bedcc"} Aug 04 09:42:00.720560 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:00.720542 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" Aug 04 09:42:00.732797 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:00.732748 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-wf7mf" podStartSLOduration=2.535317751 podStartE2EDuration="3.732710251s" totalImagesPullingTime="1.1973925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:42:00.731026619 +0000 UTC m=+89.922319121" watchObservedRunningTime="2026-08-04 09:42:00.732710251 +0000 UTC m=+89.924002746" Aug 04 09:42:00.752998 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:00.752955 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" podStartSLOduration=50.64084875 podStartE2EDuration="53.752940418s" totalImagesPullingTime="3.112091668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:42:00.75228791 +0000 UTC m=+89.943580405" watchObservedRunningTime="2026-08-04 09:42:00.752940418 +0000 UTC m=+89.944232914" Aug 04 09:42:00.991566 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:00.991544 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-t2tlf" Aug 04 09:42:01.723212 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:01.723172 2575 generic.go:356] "Generic (PLEG): container finished" podID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerID="d5eb2a9d13da2c9024a90c89003742798fbeaad96589a27b9345ecdf99958224" exitCode=0 Aug 04 09:42:01.723586 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:01.723256 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerDied","Data":"d5eb2a9d13da2c9024a90c89003742798fbeaad96589a27b9345ecdf99958224"} Aug 04 09:42:05.724309 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:05.724271 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6964db96b7-vfn66" Aug 04 09:42:05.742983 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:05.742955 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerStarted","Data":"1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6"} Aug 04 09:42:05.743113 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:05.743012 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerStarted","Data":"659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1"} Aug 04 09:42:05.743113 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:05.743039 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerStarted","Data":"43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f"} Aug 04 09:42:05.743113 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:05.743061 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerStarted","Data":"977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0"} Aug 04 09:42:05.743113 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:05.743086 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerStarted","Data":"1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d"} Aug 04 09:42:05.743113 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:05.743112 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerStarted","Data":"d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041"} Aug 04 09:42:05.806092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:05.806030 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.757988265 podStartE2EDuration="6.806007024s" totalImagesPullingTime="4.027645531s" totalInitContainerRuntime="20.373228ms" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:42:05.803532262 +0000 UTC m=+94.994824803" watchObservedRunningTime="2026-08-04 09:42:05.806007024 +0000 UTC m=+94.997299522" Aug 04 09:42:08.582514 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:08.582476 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c6bvn" Aug 04 09:42:09.763479 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:09.763447 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:42:42.891751 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:42.891651 2575 generic.go:356] "Generic (PLEG): container finished" podID="bbfc0fc9-940f-4c54-a66d-ac422afa0ceb" containerID="00992235cf15be8f906bb0f2d083e5df88fc1113d5e01609203c06a3373c3ed1" exitCode=0 Aug 04 09:42:42.892110 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:42.891749 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-b5jm2" event={"ID":"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb","Type":"ContainerDied","Data":"00992235cf15be8f906bb0f2d083e5df88fc1113d5e01609203c06a3373c3ed1"} Aug 04 09:42:42.892151 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:42.892118 2575 scope.go:117] "RemoveContainer" containerID="00992235cf15be8f906bb0f2d083e5df88fc1113d5e01609203c06a3373c3ed1" Aug 04 09:42:43.896877 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:43.896842 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-b5jm2" event={"ID":"bbfc0fc9-940f-4c54-a66d-ac422afa0ceb","Type":"ContainerStarted","Data":"29300ee9849840168a390a986c9be53af7d9eff2999c886a53649748e4946b9f"} Aug 04 09:42:46.912136 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:46.912095 2575 generic.go:356] "Generic (PLEG): container finished" podID="f4120302-68f5-4b8b-b2bf-fc38adebec10" containerID="8147a414c7a7bffaa1bfe574d31545bd5d4b387076149652e9717cba4f2f0da1" exitCode=0 Aug 04 09:42:46.912550 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:46.912132 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" event={"ID":"f4120302-68f5-4b8b-b2bf-fc38adebec10","Type":"ContainerDied","Data":"8147a414c7a7bffaa1bfe574d31545bd5d4b387076149652e9717cba4f2f0da1"} Aug 04 09:42:46.912550 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:46.912438 2575 scope.go:117] "RemoveContainer" containerID="8147a414c7a7bffaa1bfe574d31545bd5d4b387076149652e9717cba4f2f0da1" Aug 04 09:42:47.917800 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:47.917756 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8hrzc" event={"ID":"f4120302-68f5-4b8b-b2bf-fc38adebec10","Type":"ContainerStarted","Data":"3e5ff5edb44da8493262a8bdd90d8186855b6a3d9a74c37219b2844623737e44"} Aug 04 09:42:59.762763 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:59.762705 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:42:59.783193 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:59.783165 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:42:59.989399 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:42:59.989366 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:18.015952 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:18.015917 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 09:43:18.016426 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:18.016193 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="prometheus" containerID="cri-o://d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041" gracePeriod=600 Aug 04 09:43:18.016426 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:18.016213 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy" containerID="cri-o://659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1" gracePeriod=600 Aug 04 09:43:18.016426 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:18.016283 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="config-reloader" containerID="cri-o://1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d" gracePeriod=600 Aug 04 09:43:18.016426 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:18.016277 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy-web" containerID="cri-o://43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f" gracePeriod=600 Aug 04 09:43:18.016426 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:18.016272 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6" gracePeriod=600 Aug 04 09:43:18.016426 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:18.016283 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="thanos-sidecar" containerID="cri-o://977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0" gracePeriod=600 Aug 04 09:43:19.063912 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.063877 2575 generic.go:356] "Generic (PLEG): container finished" podID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerID="1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6" exitCode=0 Aug 04 09:43:19.063912 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.063906 2575 generic.go:356] "Generic (PLEG): container finished" podID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerID="659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1" exitCode=0 Aug 04 09:43:19.063912 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.063913 2575 generic.go:356] "Generic (PLEG): container finished" podID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerID="977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0" exitCode=0 Aug 04 09:43:19.063912 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.063919 2575 generic.go:356] "Generic (PLEG): container finished" podID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerID="1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d" exitCode=0 Aug 04 09:43:19.063912 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.063924 2575 generic.go:356] "Generic (PLEG): container finished" podID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerID="d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041" exitCode=0 Aug 04 09:43:19.064485 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.063955 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerDied","Data":"1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6"} Aug 04 09:43:19.064485 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.064030 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerDied","Data":"659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1"} Aug 04 09:43:19.064485 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.064059 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerDied","Data":"977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0"} Aug 04 09:43:19.064485 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.064086 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerDied","Data":"1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d"} Aug 04 09:43:19.064485 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.064111 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerDied","Data":"d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041"} Aug 04 09:43:19.269202 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.269179 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:19.305370 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.305325 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-db\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.305489 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.305384 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-web-config\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.305489 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.305421 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config-out\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.305489 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.305455 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-tls\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.305489 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.305484 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-tls-assets\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.305692 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.305521 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-1\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.305692 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.305546 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-0\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.305692 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.305566 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-serving-certs-ca-bundle\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.305917 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.305975 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306102 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-kubelet-serving-ca-bundle\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306148 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-metrics-client-ca\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306366 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306427 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-metrics-client-certs\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306461 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-2\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306474 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306487 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306520 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306555 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-trusted-ca-bundle\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306578 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-thanos-prometheus-http-client-file\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306615 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-grpc-tls\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306640 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jxc2\" (UniqueName: \"kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-kube-api-access-8jxc2\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306670 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-kube-rbac-proxy\") pod \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\" (UID: \"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8\") " Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306890 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:43:19.309040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306941 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.309950 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306961 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.309950 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.306977 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.309950 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.307295 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:43:19.309950 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.308117 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:43:19.309950 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.308332 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:43:19.309950 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.308807 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:43:19.309950 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.309429 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config-out" (OuterVolumeSpecName: "config-out") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:43:19.309950 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.309478 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:43:19.309950 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.309507 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:19.310377 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.310293 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:19.311218 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.311179 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:19.311294 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.311218 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:19.311709 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.311554 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:19.312205 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.312172 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:19.312390 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.312360 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-kube-api-access-8jxc2" (OuterVolumeSpecName: "kube-api-access-8jxc2") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "kube-api-access-8jxc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:43:19.312605 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.312579 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:19.312909 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.312879 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config" (OuterVolumeSpecName: "config") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:19.324765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.324744 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-web-config" (OuterVolumeSpecName: "web-config") pod "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" (UID: "4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:19.407930 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.407910 2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-metrics-client-certs\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.407930 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.407931 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.407941 2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.407950 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.407959 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.407967 2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.407975 2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-grpc-tls\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.407984 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jxc2\" (UniqueName: \"kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-kube-api-access-8jxc2\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.407992 2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-kube-rbac-proxy\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.408000 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-db\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.408009 2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-web-config\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.408017 2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-config-out\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.408025 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.408032 2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-tls-assets\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.408040 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.408049 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-configmap-metrics-client-ca\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:19.408092 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:19.408058 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:43:20.071197 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.071158 2575 generic.go:356] "Generic (PLEG): container finished" podID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerID="43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f" exitCode=0 Aug 04 09:43:20.071555 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.071210 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerDied","Data":"43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f"} Aug 04 09:43:20.071555 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.071264 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.071555 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.071330 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8","Type":"ContainerDied","Data":"809544c1a512295e861052aa87cf0cde6b2dbbf595832d7559948c65015bedcc"} Aug 04 09:43:20.071555 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.071383 2575 scope.go:117] "RemoveContainer" containerID="1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6" Aug 04 09:43:20.079940 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.079924 2575 scope.go:117] "RemoveContainer" containerID="659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1" Aug 04 09:43:20.086503 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.086489 2575 scope.go:117] "RemoveContainer" containerID="43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f" Aug 04 09:43:20.092629 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.092616 2575 scope.go:117] "RemoveContainer" containerID="977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0" Aug 04 09:43:20.099358 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.099338 2575 scope.go:117] "RemoveContainer" containerID="1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d" Aug 04 09:43:20.100693 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.100678 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 09:43:20.102225 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.102050 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 09:43:20.106601 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.106586 2575 scope.go:117] "RemoveContainer" containerID="d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041" Aug 04 09:43:20.113312 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.113287 2575 scope.go:117] "RemoveContainer" containerID="d5eb2a9d13da2c9024a90c89003742798fbeaad96589a27b9345ecdf99958224" Aug 04 09:43:20.119372 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.119356 2575 scope.go:117] "RemoveContainer" containerID="1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6" Aug 04 09:43:20.119643 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:43:20.119622 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6\": container with ID starting with 1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6 not found: ID does not exist" containerID="1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6" Aug 04 09:43:20.119709 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.119650 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6"} err="failed to get container status \"1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6\": rpc error: code = NotFound desc = could not find container \"1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6\": container with ID starting with 1de3218f145c0e9207fce286bbe827dd7980ffc0158c231b861a3e46f11107d6 not found: ID does not exist" Aug 04 09:43:20.119709 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.119682 2575 scope.go:117] "RemoveContainer" containerID="659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1" Aug 04 09:43:20.119922 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:43:20.119907 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1\": container with ID starting with 659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1 not found: ID does not exist" containerID="659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1" Aug 04 09:43:20.119968 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.119924 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1"} err="failed to get container status \"659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1\": rpc error: code = NotFound desc = could not find container \"659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1\": container with ID starting with 659fa20032c0ae93a9e797f25184253a452ec7cb50b31e1c858423ce8d54a9a1 not found: ID does not exist" Aug 04 09:43:20.119968 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.119937 2575 scope.go:117] "RemoveContainer" containerID="43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f" Aug 04 09:43:20.120162 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:43:20.120142 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f\": container with ID starting with 43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f not found: ID does not exist" containerID="43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f" Aug 04 09:43:20.120199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.120168 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f"} err="failed to get container status \"43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f\": rpc error: code = NotFound desc = could not find container \"43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f\": container with ID starting with 43bf524c9fc8a6fafaf0b0f3fe89b450a9301e095bc5dd850ff60865b829115f not found: ID does not exist" Aug 04 09:43:20.120199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.120183 2575 scope.go:117] "RemoveContainer" containerID="977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0" Aug 04 09:43:20.120409 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:43:20.120393 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0\": container with ID starting with 977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0 not found: ID does not exist" containerID="977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0" Aug 04 09:43:20.120451 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.120414 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0"} err="failed to get container status \"977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0\": rpc error: code = NotFound desc = could not find container \"977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0\": container with ID starting with 977ccc0a5b7dd2b5ad0ad94faa518273c06545bee622c1f8d6acf777309becd0 not found: ID does not exist" Aug 04 09:43:20.120451 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.120428 2575 scope.go:117] "RemoveContainer" containerID="1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d" Aug 04 09:43:20.120621 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:43:20.120607 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d\": container with ID starting with 1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d not found: ID does not exist" containerID="1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d" Aug 04 09:43:20.120657 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.120624 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d"} err="failed to get container status \"1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d\": rpc error: code = NotFound desc = could not find container \"1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d\": container with ID starting with 1ae65fc8b2b12edc6f7aacab23b1db6efc70216de66c914f91aef395382fbc2d not found: ID does not exist" Aug 04 09:43:20.120657 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.120636 2575 scope.go:117] "RemoveContainer" containerID="d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041" Aug 04 09:43:20.120863 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:43:20.120847 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041\": container with ID starting with d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041 not found: ID does not exist" containerID="d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041" Aug 04 09:43:20.120918 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.120867 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041"} err="failed to get container status \"d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041\": rpc error: code = NotFound desc = could not find container \"d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041\": container with ID starting with d78f51a48d4ec62344f8d96577d0e965a37e05125323154bd1c7c565666b2041 not found: ID does not exist" Aug 04 09:43:20.120918 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.120880 2575 scope.go:117] "RemoveContainer" containerID="d5eb2a9d13da2c9024a90c89003742798fbeaad96589a27b9345ecdf99958224" Aug 04 09:43:20.121104 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:43:20.121081 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5eb2a9d13da2c9024a90c89003742798fbeaad96589a27b9345ecdf99958224\": container with ID starting with d5eb2a9d13da2c9024a90c89003742798fbeaad96589a27b9345ecdf99958224 not found: ID does not exist" containerID="d5eb2a9d13da2c9024a90c89003742798fbeaad96589a27b9345ecdf99958224" Aug 04 09:43:20.121147 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.121105 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5eb2a9d13da2c9024a90c89003742798fbeaad96589a27b9345ecdf99958224"} err="failed to get container status \"d5eb2a9d13da2c9024a90c89003742798fbeaad96589a27b9345ecdf99958224\": rpc error: code = NotFound desc = could not find container \"d5eb2a9d13da2c9024a90c89003742798fbeaad96589a27b9345ecdf99958224\": container with ID starting with d5eb2a9d13da2c9024a90c89003742798fbeaad96589a27b9345ecdf99958224 not found: ID does not exist" Aug 04 09:43:20.146746 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.146698 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 09:43:20.147055 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147040 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy" Aug 04 09:43:20.147130 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147057 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy" Aug 04 09:43:20.147130 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147075 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="prometheus" Aug 04 09:43:20.147130 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147084 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="prometheus" Aug 04 09:43:20.147130 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147107 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="thanos-sidecar" Aug 04 09:43:20.147130 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147115 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="thanos-sidecar" Aug 04 09:43:20.147130 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147124 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy-thanos" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147133 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy-thanos" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147148 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy-web" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147158 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy-web" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147172 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="config-reloader" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147180 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="config-reloader" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147191 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="init-config-reloader" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147200 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="init-config-reloader" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147290 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy-web" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147304 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="config-reloader" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147313 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="prometheus" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147321 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="thanos-sidecar" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147329 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy-thanos" Aug 04 09:43:20.147400 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.147337 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" containerName="kube-rbac-proxy" Aug 04 09:43:20.152640 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.152624 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.157246 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.157229 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Aug 04 09:43:20.157680 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.157665 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Aug 04 09:43:20.157741 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.157674 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Aug 04 09:43:20.157824 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.157671 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Aug 04 09:43:20.159853 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.159821 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Aug 04 09:43:20.159853 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.159842 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Aug 04 09:43:20.160002 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.159913 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Aug 04 09:43:20.160002 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.159925 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Aug 04 09:43:20.160002 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.159842 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Aug 04 09:43:20.160142 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.160128 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-w88rc\"" Aug 04 09:43:20.160449 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.160414 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Aug 04 09:43:20.160588 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.160421 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Aug 04 09:43:20.160698 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.160682 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a0ofjc2sesvat\"" Aug 04 09:43:20.160775 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.160687 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Aug 04 09:43:20.166382 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.166359 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Aug 04 09:43:20.168525 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.168507 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Aug 04 09:43:20.168881 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.168866 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 09:43:20.172417 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.172399 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Aug 04 09:43:20.214142 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214116 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214235 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214145 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214235 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214163 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-config\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214235 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214179 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214235 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214235 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214225 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214454 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214249 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214454 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214267 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214454 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214301 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214454 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214334 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214454 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214350 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4sgk\" (UniqueName: \"kubernetes.io/projected/f414f234-fd1f-4726-b761-027122ab5cd6-kube-api-access-t4sgk\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214454 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214367 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f414f234-fd1f-4726-b761-027122ab5cd6-config-out\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214454 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214403 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214454 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214453 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f414f234-fd1f-4726-b761-027122ab5cd6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214479 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214512 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214538 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214561 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-web-config\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214589 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.214765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.214622 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315116 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315093 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315116 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315118 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4sgk\" (UniqueName: \"kubernetes.io/projected/f414f234-fd1f-4726-b761-027122ab5cd6-kube-api-access-t4sgk\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315256 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315135 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f414f234-fd1f-4726-b761-027122ab5cd6-config-out\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315256 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315152 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315256 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315167 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f414f234-fd1f-4726-b761-027122ab5cd6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315256 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315189 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315256 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315218 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315256 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315244 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315270 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-web-config\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315298 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315321 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315359 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315386 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315413 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-config\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315438 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315476 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315501 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.315535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315505 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.316021 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315537 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.316021 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315585 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.316021 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.315614 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.316021 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.316010 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.316207 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.316116 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.317607 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.316672 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.317607 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.317333 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.317862 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.317839 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.318489 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.318285 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.318962 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.318738 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.318962 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.318753 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.319950 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.319894 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f414f234-fd1f-4726-b761-027122ab5cd6-config-out\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.319950 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.319910 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.320247 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.320104 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.320247 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.320185 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.320420 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.320324 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f414f234-fd1f-4726-b761-027122ab5cd6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.320537 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.320518 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-web-config\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.320577 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.320554 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-config\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.320719 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.320696 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.320988 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.320971 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f414f234-fd1f-4726-b761-027122ab5cd6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.321665 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.321628 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f414f234-fd1f-4726-b761-027122ab5cd6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.331548 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.331529 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4sgk\" (UniqueName: \"kubernetes.io/projected/f414f234-fd1f-4726-b761-027122ab5cd6-kube-api-access-t4sgk\") pod \"prometheus-k8s-0\" (UID: \"f414f234-fd1f-4726-b761-027122ab5cd6\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.462426 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.462396 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:43:20.607592 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:20.607569 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 09:43:20.609338 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:43:20.609311 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf414f234_fd1f_4726_b761_027122ab5cd6.slice/crio-b12fc9a6a821ea5a4005a6fbd2684be78e23ac931cf643ac6ad28dacc7f7f1ee WatchSource:0}: Error finding container b12fc9a6a821ea5a4005a6fbd2684be78e23ac931cf643ac6ad28dacc7f7f1ee: Status 404 returned error can't find the container with id b12fc9a6a821ea5a4005a6fbd2684be78e23ac931cf643ac6ad28dacc7f7f1ee Aug 04 09:43:21.077237 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:21.077202 2575 generic.go:356] "Generic (PLEG): container finished" podID="f414f234-fd1f-4726-b761-027122ab5cd6" containerID="a8e50605e8cd75c87ffb41e39f5d0d35a9245c693af51300a7fae52f9001ab27" exitCode=0 Aug 04 09:43:21.077565 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:21.077290 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f414f234-fd1f-4726-b761-027122ab5cd6","Type":"ContainerDied","Data":"a8e50605e8cd75c87ffb41e39f5d0d35a9245c693af51300a7fae52f9001ab27"} Aug 04 09:43:21.077565 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:21.077377 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f414f234-fd1f-4726-b761-027122ab5cd6","Type":"ContainerStarted","Data":"b12fc9a6a821ea5a4005a6fbd2684be78e23ac931cf643ac6ad28dacc7f7f1ee"} Aug 04 09:43:21.338981 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:21.338952 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8" path="/var/lib/kubelet/pods/4102bfb7-9ff3-41d4-af13-5a5f9a45d0c8/volumes" Aug 04 09:43:22.087448 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:22.087413 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f414f234-fd1f-4726-b761-027122ab5cd6","Type":"ContainerStarted","Data":"0482dd03d16107ed9455a989042d659f2e415c2b96636e69245fc008f197e5de"} Aug 04 09:43:22.087834 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:22.087480 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f414f234-fd1f-4726-b761-027122ab5cd6","Type":"ContainerStarted","Data":"58a4bfe880b9655f1f1613a14aa946d432cf5688e3366769f361c4c6e3d44fb0"} Aug 04 09:43:22.087834 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:22.087509 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f414f234-fd1f-4726-b761-027122ab5cd6","Type":"ContainerStarted","Data":"93314356acb4ec8d7905099b797c80b78155690548175f579cd1ed3c5c80d9d0"} Aug 04 09:43:22.087834 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:22.087537 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f414f234-fd1f-4726-b761-027122ab5cd6","Type":"ContainerStarted","Data":"7ea3871dad36393c09092bdc4a9846fdcf006e269f91ea6820fa15ba37dbcb30"} Aug 04 09:43:22.087834 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:22.087567 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f414f234-fd1f-4726-b761-027122ab5cd6","Type":"ContainerStarted","Data":"38660e957bf7b7b61dfd3d3126cf18285256614853a29f70d1bc701d42b92157"} Aug 04 09:43:22.087834 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:22.087596 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f414f234-fd1f-4726-b761-027122ab5cd6","Type":"ContainerStarted","Data":"19005348ee0a99057987dd8fe0ccf3fb0ff99256f462f63069d88f68db6d6853"} Aug 04 09:43:22.123081 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:22.123032 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.105841198 podStartE2EDuration="2.123012266s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.171068ms" isStatefulPod=false podCreationTimestamp="2026-08-04 09:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:43:22.120198612 +0000 UTC m=+171.311491107" watchObservedRunningTime="2026-08-04 09:43:22.123012266 +0000 UTC m=+171.314304838" Aug 04 09:43:25.462574 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:43:25.462501 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:44:20.463594 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:44:20.463544 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:44:20.478529 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:44:20.478507 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:44:21.358678 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:44:21.358651 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 09:45:05.237926 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.237890 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct"] Aug 04 09:45:05.241377 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.241354 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct" Aug 04 09:45:05.245007 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.244985 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-7nskg\"" Aug 04 09:45:05.245093 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.245072 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:45:05.245610 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.245595 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Aug 04 09:45:05.254649 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.254631 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct"] Aug 04 09:45:05.357016 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.356991 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17f2b900-d774-4885-a6c9-9e94a9b207af-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xrbct\" (UID: \"17f2b900-d774-4885-a6c9-9e94a9b207af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct" Aug 04 09:45:05.357107 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.357032 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlzhx\" (UniqueName: \"kubernetes.io/projected/17f2b900-d774-4885-a6c9-9e94a9b207af-kube-api-access-nlzhx\") pod \"openshift-lws-operator-fd8ccff4c-xrbct\" (UID: \"17f2b900-d774-4885-a6c9-9e94a9b207af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct" Aug 04 09:45:05.458336 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.458307 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlzhx\" (UniqueName: \"kubernetes.io/projected/17f2b900-d774-4885-a6c9-9e94a9b207af-kube-api-access-nlzhx\") pod \"openshift-lws-operator-fd8ccff4c-xrbct\" (UID: \"17f2b900-d774-4885-a6c9-9e94a9b207af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct" Aug 04 09:45:05.458481 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.458399 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17f2b900-d774-4885-a6c9-9e94a9b207af-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xrbct\" (UID: \"17f2b900-d774-4885-a6c9-9e94a9b207af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct" Aug 04 09:45:05.458714 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.458695 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17f2b900-d774-4885-a6c9-9e94a9b207af-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xrbct\" (UID: \"17f2b900-d774-4885-a6c9-9e94a9b207af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct" Aug 04 09:45:05.474621 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.474586 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlzhx\" (UniqueName: \"kubernetes.io/projected/17f2b900-d774-4885-a6c9-9e94a9b207af-kube-api-access-nlzhx\") pod \"openshift-lws-operator-fd8ccff4c-xrbct\" (UID: \"17f2b900-d774-4885-a6c9-9e94a9b207af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct" Aug 04 09:45:05.559483 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.559428 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct" Aug 04 09:45:05.689260 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:05.689219 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct"] Aug 04 09:45:05.692645 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:45:05.692614 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17f2b900_d774_4885_a6c9_9e94a9b207af.slice/crio-36eb9144c9362b0e0cf242a07b29bbe1e2317803aff04547ecd9673cfb6c1ddc WatchSource:0}: Error finding container 36eb9144c9362b0e0cf242a07b29bbe1e2317803aff04547ecd9673cfb6c1ddc: Status 404 returned error can't find the container with id 36eb9144c9362b0e0cf242a07b29bbe1e2317803aff04547ecd9673cfb6c1ddc Aug 04 09:45:06.528552 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:06.528507 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct" event={"ID":"17f2b900-d774-4885-a6c9-9e94a9b207af","Type":"ContainerStarted","Data":"36eb9144c9362b0e0cf242a07b29bbe1e2317803aff04547ecd9673cfb6c1ddc"} Aug 04 09:45:08.539542 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:08.539507 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct" event={"ID":"17f2b900-d774-4885-a6c9-9e94a9b207af","Type":"ContainerStarted","Data":"85c7cb5982f6f31772506b56bbb368ccc7a5db985e10227d818fdde32ff35ce1"} Aug 04 09:45:23.689527 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.689473 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrbct" podStartSLOduration=16.257630708 podStartE2EDuration="18.689455959s" totalImagesPullingTime="2.431825251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:45:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:45:08.565853375 +0000 UTC m=+277.757145872" watchObservedRunningTime="2026-08-04 09:45:23.689455959 +0000 UTC m=+292.880748454" Aug 04 09:45:23.690626 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.690605 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc"] Aug 04 09:45:23.693816 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.693799 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.696678 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.696652 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b41d453-da85-4848-b88c-8228f8869f83-cert\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.696678 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.696688 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2b41d453-da85-4848-b88c-8228f8869f83-manager-config\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.696874 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.696741 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z48sw\" (UniqueName: \"kubernetes.io/projected/2b41d453-da85-4848-b88c-8228f8869f83-kube-api-access-z48sw\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.696874 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.696842 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b41d453-da85-4848-b88c-8228f8869f83-metrics-cert\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.699587 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.699554 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Aug 04 09:45:23.699738 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.699599 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Aug 04 09:45:23.699738 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.699628 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-t997z\"" Aug 04 09:45:23.724462 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.724432 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Aug 04 09:45:23.737978 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.737946 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc"] Aug 04 09:45:23.797472 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.797431 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b41d453-da85-4848-b88c-8228f8869f83-metrics-cert\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.797666 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.797483 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b41d453-da85-4848-b88c-8228f8869f83-cert\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.797666 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.797506 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2b41d453-da85-4848-b88c-8228f8869f83-manager-config\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.797666 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.797537 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z48sw\" (UniqueName: \"kubernetes.io/projected/2b41d453-da85-4848-b88c-8228f8869f83-kube-api-access-z48sw\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.798309 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.798272 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2b41d453-da85-4848-b88c-8228f8869f83-manager-config\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.799953 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.799924 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b41d453-da85-4848-b88c-8228f8869f83-cert\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.800054 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.800011 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b41d453-da85-4848-b88c-8228f8869f83-metrics-cert\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:23.836096 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:23.836058 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z48sw\" (UniqueName: \"kubernetes.io/projected/2b41d453-da85-4848-b88c-8228f8869f83-kube-api-access-z48sw\") pod \"lws-controller-manager-78b8bf7c54-nkwzc\" (UID: \"2b41d453-da85-4848-b88c-8228f8869f83\") " pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:24.003493 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:24.003458 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:24.157913 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:24.157883 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc"] Aug 04 09:45:24.159798 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:45:24.159773 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b41d453_da85_4848_b88c_8228f8869f83.slice/crio-54e13d6f94ba05eaf92ee4579e37ecf8ecf4e102988b30c9823c75a189a5fe4f WatchSource:0}: Error finding container 54e13d6f94ba05eaf92ee4579e37ecf8ecf4e102988b30c9823c75a189a5fe4f: Status 404 returned error can't find the container with id 54e13d6f94ba05eaf92ee4579e37ecf8ecf4e102988b30c9823c75a189a5fe4f Aug 04 09:45:24.603616 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:24.603576 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" event={"ID":"2b41d453-da85-4848-b88c-8228f8869f83","Type":"ContainerStarted","Data":"54e13d6f94ba05eaf92ee4579e37ecf8ecf4e102988b30c9823c75a189a5fe4f"} Aug 04 09:45:25.265457 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.265422 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt"] Aug 04 09:45:25.270164 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.270133 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.273924 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.273903 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 04 09:45:25.274210 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.274197 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 04 09:45:25.274455 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.274439 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 04 09:45:25.274581 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.274563 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9nfsq\"" Aug 04 09:45:25.274626 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.274563 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 04 09:45:25.306110 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.306087 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt"] Aug 04 09:45:25.312403 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.312364 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-manifests\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.312514 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.312425 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.312569 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.312527 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-tmp\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.312673 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.312596 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.312673 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.312649 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plct7\" (UniqueName: \"kubernetes.io/projected/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-kube-api-access-plct7\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.414105 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.414071 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.414280 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.414134 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plct7\" (UniqueName: \"kubernetes.io/projected/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-kube-api-access-plct7\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.414280 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.414161 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-manifests\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.414280 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.414264 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.414411 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.414351 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-tmp\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.414591 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.414561 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-manifests\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.416427 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.416402 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-tmp\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.416520 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.416485 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.416565 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.416515 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.447580 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.447557 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plct7\" (UniqueName: \"kubernetes.io/projected/99873edc-d283-4a3c-8033-6b6c8cd9f9a4-kube-api-access-plct7\") pod \"opendatahub-operator-controller-manager-7bb649b865-qptgt\" (UID: \"99873edc-d283-4a3c-8033-6b6c8cd9f9a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.580889 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.580810 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:25.767248 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:25.767221 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt"] Aug 04 09:45:25.769390 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:45:25.769363 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99873edc_d283_4a3c_8033_6b6c8cd9f9a4.slice/crio-76710a7ac6ad119378b5cb64c7239e96df8dbf3877e609bba84ee5f5fe746eca WatchSource:0}: Error finding container 76710a7ac6ad119378b5cb64c7239e96df8dbf3877e609bba84ee5f5fe746eca: Status 404 returned error can't find the container with id 76710a7ac6ad119378b5cb64c7239e96df8dbf3877e609bba84ee5f5fe746eca Aug 04 09:45:26.618317 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:26.618233 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" event={"ID":"99873edc-d283-4a3c-8033-6b6c8cd9f9a4","Type":"ContainerStarted","Data":"76710a7ac6ad119378b5cb64c7239e96df8dbf3877e609bba84ee5f5fe746eca"} Aug 04 09:45:26.619878 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:26.619850 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" event={"ID":"2b41d453-da85-4848-b88c-8228f8869f83","Type":"ContainerStarted","Data":"7b3ba9e005e3e7b6ba420ffd05b0e2928ca8ce4b7289d579598f4a05cea1adfa"} Aug 04 09:45:26.620017 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:26.619925 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:29.637485 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:29.637449 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" event={"ID":"99873edc-d283-4a3c-8033-6b6c8cd9f9a4","Type":"ContainerStarted","Data":"92e764a9efdba15887ada305daa5c2c32e43a92ae96b4fd90d4152a084c2f627"} Aug 04 09:45:29.673208 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:29.673162 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" podStartSLOduration=4.539627751 podStartE2EDuration="6.673146811s" totalImagesPullingTime="2.13351906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:45:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:45:26.662180117 +0000 UTC m=+295.853472611" watchObservedRunningTime="2026-08-04 09:45:29.673146811 +0000 UTC m=+298.864439308" Aug 04 09:45:30.642457 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:30.642414 2575 generic.go:356] "Generic (PLEG): container finished" podID="99873edc-d283-4a3c-8033-6b6c8cd9f9a4" containerID="92e764a9efdba15887ada305daa5c2c32e43a92ae96b4fd90d4152a084c2f627" exitCode=0 Aug 04 09:45:30.642866 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:30.642496 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" event={"ID":"99873edc-d283-4a3c-8033-6b6c8cd9f9a4","Type":"ContainerDied","Data":"92e764a9efdba15887ada305daa5c2c32e43a92ae96b4fd90d4152a084c2f627"} Aug 04 09:45:31.161061 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:31.161035 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 09:45:31.173213 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:31.173180 2575 kubelet.go:1648] "Image garbage collection succeeded" Aug 04 09:45:31.178801 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:31.178779 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 09:45:31.651786 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:31.651745 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" event={"ID":"99873edc-d283-4a3c-8033-6b6c8cd9f9a4","Type":"ContainerStarted","Data":"43357f52e43b7f627e60a6fd85e193874622d065be7475923f3cd95daba134c2"} Aug 04 09:45:31.654199 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:31.651873 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:31.679068 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:31.679025 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" podStartSLOduration=2.421010067 podStartE2EDuration="6.679010248s" totalImagesPullingTime="3.252261983s" totalInitContainerRuntime="1.005738198s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:45:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:45:31.678951269 +0000 UTC m=+300.870243763" watchObservedRunningTime="2026-08-04 09:45:31.679010248 +0000 UTC m=+300.870302742" Aug 04 09:45:37.630262 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:37.630232 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78b8bf7c54-nkwzc" Aug 04 09:45:42.659586 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:42.659534 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-qptgt" Aug 04 09:45:52.076048 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.076009 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj"] Aug 04 09:45:52.086636 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.086614 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" Aug 04 09:45:52.092676 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.092650 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Aug 04 09:45:52.093282 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.092773 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-w9mff\"" Aug 04 09:45:52.093446 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.092993 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 04 09:45:52.093446 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.093413 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 04 09:45:52.093565 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.093466 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Aug 04 09:45:52.102121 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.102093 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj"] Aug 04 09:45:52.150762 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.150718 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d14c47a-670f-4850-be5c-fe5f43a8a525-tls-certs\") pod \"kube-auth-proxy-b474cb6fc-kb2gj\" (UID: \"2d14c47a-670f-4850-be5c-fe5f43a8a525\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" Aug 04 09:45:52.150867 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.150768 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jg7h\" (UniqueName: \"kubernetes.io/projected/2d14c47a-670f-4850-be5c-fe5f43a8a525-kube-api-access-2jg7h\") pod \"kube-auth-proxy-b474cb6fc-kb2gj\" (UID: \"2d14c47a-670f-4850-be5c-fe5f43a8a525\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" Aug 04 09:45:52.150867 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.150844 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d14c47a-670f-4850-be5c-fe5f43a8a525-tmp\") pod \"kube-auth-proxy-b474cb6fc-kb2gj\" (UID: \"2d14c47a-670f-4850-be5c-fe5f43a8a525\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" Aug 04 09:45:52.251835 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.251812 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d14c47a-670f-4850-be5c-fe5f43a8a525-tls-certs\") pod \"kube-auth-proxy-b474cb6fc-kb2gj\" (UID: \"2d14c47a-670f-4850-be5c-fe5f43a8a525\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" Aug 04 09:45:52.251948 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.251844 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jg7h\" (UniqueName: \"kubernetes.io/projected/2d14c47a-670f-4850-be5c-fe5f43a8a525-kube-api-access-2jg7h\") pod \"kube-auth-proxy-b474cb6fc-kb2gj\" (UID: \"2d14c47a-670f-4850-be5c-fe5f43a8a525\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" Aug 04 09:45:52.252003 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.251982 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d14c47a-670f-4850-be5c-fe5f43a8a525-tmp\") pod \"kube-auth-proxy-b474cb6fc-kb2gj\" (UID: \"2d14c47a-670f-4850-be5c-fe5f43a8a525\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" Aug 04 09:45:52.254074 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.254056 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d14c47a-670f-4850-be5c-fe5f43a8a525-tmp\") pod \"kube-auth-proxy-b474cb6fc-kb2gj\" (UID: \"2d14c47a-670f-4850-be5c-fe5f43a8a525\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" Aug 04 09:45:52.254174 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.254159 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d14c47a-670f-4850-be5c-fe5f43a8a525-tls-certs\") pod \"kube-auth-proxy-b474cb6fc-kb2gj\" (UID: \"2d14c47a-670f-4850-be5c-fe5f43a8a525\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" Aug 04 09:45:52.263507 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.263489 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jg7h\" (UniqueName: \"kubernetes.io/projected/2d14c47a-670f-4850-be5c-fe5f43a8a525-kube-api-access-2jg7h\") pod \"kube-auth-proxy-b474cb6fc-kb2gj\" (UID: \"2d14c47a-670f-4850-be5c-fe5f43a8a525\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" Aug 04 09:45:52.396286 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.396220 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" Aug 04 09:45:52.525195 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.525169 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj"] Aug 04 09:45:52.527487 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:45:52.527459 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d14c47a_670f_4850_be5c_fe5f43a8a525.slice/crio-ba6a77a84d5baa072121a12ef09ed88e88ae7527afc15930776983bb74557f75 WatchSource:0}: Error finding container ba6a77a84d5baa072121a12ef09ed88e88ae7527afc15930776983bb74557f75: Status 404 returned error can't find the container with id ba6a77a84d5baa072121a12ef09ed88e88ae7527afc15930776983bb74557f75 Aug 04 09:45:52.529266 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.529249 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 09:45:52.745694 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:52.745660 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" event={"ID":"2d14c47a-670f-4850-be5c-fe5f43a8a525","Type":"ContainerStarted","Data":"ba6a77a84d5baa072121a12ef09ed88e88ae7527afc15930776983bb74557f75"} Aug 04 09:45:55.763113 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:45:55.763080 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" event={"ID":"2d14c47a-670f-4850-be5c-fe5f43a8a525","Type":"ContainerStarted","Data":"798b84e48e777ec2610460e8cd3b13aa4e352d14b4707af94e223e1f85c278a0"} Aug 04 09:47:49.267949 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.267891 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-b474cb6fc-kb2gj" podStartSLOduration=114.393582166 podStartE2EDuration="1m57.267875554s" totalImagesPullingTime="2.874293388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:45:55.799518567 +0000 UTC m=+324.990811062" watchObservedRunningTime="2026-08-04 09:47:49.267875554 +0000 UTC m=+438.459168048" Aug 04 09:47:49.268978 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.268951 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w"] Aug 04 09:47:49.272496 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.272480 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:47:49.285426 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.285413 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 09:47:49.285929 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.285913 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 09:47:49.286029 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.285963 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wp5hm\"" Aug 04 09:47:49.359935 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.359905 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a36a5433-e114-4200-a836-e654c0cb8b30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-8g79w\" (UID: \"a36a5433-e114-4200-a836-e654c0cb8b30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:47:49.359935 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.359937 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n24b\" (UniqueName: \"kubernetes.io/projected/a36a5433-e114-4200-a836-e654c0cb8b30-kube-api-access-9n24b\") pod \"kuadrant-operator-controller-manager-58fc56757c-8g79w\" (UID: \"a36a5433-e114-4200-a836-e654c0cb8b30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:47:49.441903 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.441874 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w"] Aug 04 09:47:49.460754 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.460718 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a36a5433-e114-4200-a836-e654c0cb8b30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-8g79w\" (UID: \"a36a5433-e114-4200-a836-e654c0cb8b30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:47:49.460856 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.460759 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n24b\" (UniqueName: \"kubernetes.io/projected/a36a5433-e114-4200-a836-e654c0cb8b30-kube-api-access-9n24b\") pod \"kuadrant-operator-controller-manager-58fc56757c-8g79w\" (UID: \"a36a5433-e114-4200-a836-e654c0cb8b30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:47:49.461137 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.461120 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a36a5433-e114-4200-a836-e654c0cb8b30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-8g79w\" (UID: \"a36a5433-e114-4200-a836-e654c0cb8b30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:47:49.480338 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.480313 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n24b\" (UniqueName: \"kubernetes.io/projected/a36a5433-e114-4200-a836-e654c0cb8b30-kube-api-access-9n24b\") pod \"kuadrant-operator-controller-manager-58fc56757c-8g79w\" (UID: \"a36a5433-e114-4200-a836-e654c0cb8b30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:47:49.582336 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.582266 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:47:49.746128 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:49.746103 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w"] Aug 04 09:47:49.749027 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:47:49.748993 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda36a5433_e114_4200_a836_e654c0cb8b30.slice/crio-60b33c614e5eeb5ee9153fa1593af1aa01f8a289f537ff6d170bb5fc1f11d957 WatchSource:0}: Error finding container 60b33c614e5eeb5ee9153fa1593af1aa01f8a289f537ff6d170bb5fc1f11d957: Status 404 returned error can't find the container with id 60b33c614e5eeb5ee9153fa1593af1aa01f8a289f537ff6d170bb5fc1f11d957 Aug 04 09:47:50.288282 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:50.288248 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" event={"ID":"a36a5433-e114-4200-a836-e654c0cb8b30","Type":"ContainerStarted","Data":"60b33c614e5eeb5ee9153fa1593af1aa01f8a289f537ff6d170bb5fc1f11d957"} Aug 04 09:47:54.310722 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:54.310678 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" event={"ID":"a36a5433-e114-4200-a836-e654c0cb8b30","Type":"ContainerStarted","Data":"dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396"} Aug 04 09:47:54.337702 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:54.337646 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" podStartSLOduration=1.642814786 podStartE2EDuration="5.337613836s" totalImagesPullingTime="3.69479905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:47:54.336243806 +0000 UTC m=+443.527536300" watchObservedRunningTime="2026-08-04 09:47:54.337613836 +0000 UTC m=+443.528906332" Aug 04 09:47:55.315263 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:47:55.315227 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:48:04.093023 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.092987 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w"] Aug 04 09:48:04.093389 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.093178 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" podUID="a36a5433-e114-4200-a836-e654c0cb8b30" containerName="manager" containerID="cri-o://dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396" gracePeriod=10 Aug 04 09:48:04.094795 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.094770 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:48:04.333976 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.333956 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:48:04.353575 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.353516 2575 generic.go:356] "Generic (PLEG): container finished" podID="a36a5433-e114-4200-a836-e654c0cb8b30" containerID="dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396" exitCode=0 Aug 04 09:48:04.353575 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.353556 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" Aug 04 09:48:04.353780 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.353595 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" event={"ID":"a36a5433-e114-4200-a836-e654c0cb8b30","Type":"ContainerDied","Data":"dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396"} Aug 04 09:48:04.353780 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.353643 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w" event={"ID":"a36a5433-e114-4200-a836-e654c0cb8b30","Type":"ContainerDied","Data":"60b33c614e5eeb5ee9153fa1593af1aa01f8a289f537ff6d170bb5fc1f11d957"} Aug 04 09:48:04.353780 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.353659 2575 scope.go:117] "RemoveContainer" containerID="dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396" Aug 04 09:48:04.361839 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.361813 2575 scope.go:117] "RemoveContainer" containerID="dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396" Aug 04 09:48:04.362118 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:48:04.362099 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396\": container with ID starting with dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396 not found: ID does not exist" containerID="dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396" Aug 04 09:48:04.362195 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.362130 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396"} err="failed to get container status \"dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396\": rpc error: code = NotFound desc = could not find container \"dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396\": container with ID starting with dfdc39e3d7702d072098ca39dd85ca3a1782a8add622eb46392ffe82448e8396 not found: ID does not exist" Aug 04 09:48:04.486076 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.486052 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9n24b\" (UniqueName: \"kubernetes.io/projected/a36a5433-e114-4200-a836-e654c0cb8b30-kube-api-access-9n24b\") pod \"a36a5433-e114-4200-a836-e654c0cb8b30\" (UID: \"a36a5433-e114-4200-a836-e654c0cb8b30\") " Aug 04 09:48:04.486218 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.486084 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a36a5433-e114-4200-a836-e654c0cb8b30-extensions-socket-volume\") pod \"a36a5433-e114-4200-a836-e654c0cb8b30\" (UID: \"a36a5433-e114-4200-a836-e654c0cb8b30\") " Aug 04 09:48:04.486599 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.486567 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a36a5433-e114-4200-a836-e654c0cb8b30-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a36a5433-e114-4200-a836-e654c0cb8b30" (UID: "a36a5433-e114-4200-a836-e654c0cb8b30"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:48:04.488241 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.488217 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a36a5433-e114-4200-a836-e654c0cb8b30-kube-api-access-9n24b" (OuterVolumeSpecName: "kube-api-access-9n24b") pod "a36a5433-e114-4200-a836-e654c0cb8b30" (UID: "a36a5433-e114-4200-a836-e654c0cb8b30"). InnerVolumeSpecName "kube-api-access-9n24b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:48:04.587549 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.587518 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n24b\" (UniqueName: \"kubernetes.io/projected/a36a5433-e114-4200-a836-e654c0cb8b30-kube-api-access-9n24b\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:48:04.587549 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.587546 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a36a5433-e114-4200-a836-e654c0cb8b30-extensions-socket-volume\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:48:04.681328 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.681300 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w"] Aug 04 09:48:04.692051 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:04.692029 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-8g79w"] Aug 04 09:48:05.338124 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:05.338091 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a36a5433-e114-4200-a836-e654c0cb8b30" path="/var/lib/kubelet/pods/a36a5433-e114-4200-a836-e654c0cb8b30/volumes" Aug 04 09:48:23.397303 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.397269 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f2wwt"] Aug 04 09:48:23.397765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.397594 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a36a5433-e114-4200-a836-e654c0cb8b30" containerName="manager" Aug 04 09:48:23.397765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.397605 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a36a5433-e114-4200-a836-e654c0cb8b30" containerName="manager" Aug 04 09:48:23.397765 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.397671 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a36a5433-e114-4200-a836-e654c0cb8b30" containerName="manager" Aug 04 09:48:23.404127 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.404109 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:23.407131 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.407110 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 09:48:23.409002 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.408976 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 09:48:23.409598 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.409561 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Aug 04 09:48:23.409798 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.409779 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-878ss\"" Aug 04 09:48:23.412421 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.412148 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f2wwt"] Aug 04 09:48:23.471859 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.471830 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f2wwt"] Aug 04 09:48:23.542701 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.542674 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbm8x\" (UniqueName: \"kubernetes.io/projected/f2a72770-b12d-42e8-9894-ad9b9f317c02-kube-api-access-sbm8x\") pod \"limitador-limitador-859c4d96d6-f2wwt\" (UID: \"f2a72770-b12d-42e8-9894-ad9b9f317c02\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:23.542858 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.542709 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f2a72770-b12d-42e8-9894-ad9b9f317c02-config-file\") pod \"limitador-limitador-859c4d96d6-f2wwt\" (UID: \"f2a72770-b12d-42e8-9894-ad9b9f317c02\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:23.643755 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.643715 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f2a72770-b12d-42e8-9894-ad9b9f317c02-config-file\") pod \"limitador-limitador-859c4d96d6-f2wwt\" (UID: \"f2a72770-b12d-42e8-9894-ad9b9f317c02\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:23.643867 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.643811 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbm8x\" (UniqueName: \"kubernetes.io/projected/f2a72770-b12d-42e8-9894-ad9b9f317c02-kube-api-access-sbm8x\") pod \"limitador-limitador-859c4d96d6-f2wwt\" (UID: \"f2a72770-b12d-42e8-9894-ad9b9f317c02\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:23.644295 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.644276 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f2a72770-b12d-42e8-9894-ad9b9f317c02-config-file\") pod \"limitador-limitador-859c4d96d6-f2wwt\" (UID: \"f2a72770-b12d-42e8-9894-ad9b9f317c02\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:23.652318 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.652261 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbm8x\" (UniqueName: \"kubernetes.io/projected/f2a72770-b12d-42e8-9894-ad9b9f317c02-kube-api-access-sbm8x\") pod \"limitador-limitador-859c4d96d6-f2wwt\" (UID: \"f2a72770-b12d-42e8-9894-ad9b9f317c02\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:23.716271 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.716249 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:23.845773 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:23.845701 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f2wwt"] Aug 04 09:48:23.847900 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:48:23.847870 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a72770_b12d_42e8_9894_ad9b9f317c02.slice/crio-11cca87eab4f7ef36645df3bf641650fc3f119f3d6b38efa689f81b571aa9398 WatchSource:0}: Error finding container 11cca87eab4f7ef36645df3bf641650fc3f119f3d6b38efa689f81b571aa9398: Status 404 returned error can't find the container with id 11cca87eab4f7ef36645df3bf641650fc3f119f3d6b38efa689f81b571aa9398 Aug 04 09:48:24.440751 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:24.440701 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" event={"ID":"f2a72770-b12d-42e8-9894-ad9b9f317c02","Type":"ContainerStarted","Data":"11cca87eab4f7ef36645df3bf641650fc3f119f3d6b38efa689f81b571aa9398"} Aug 04 09:48:24.675586 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:24.675552 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d9jxz"] Aug 04 09:48:24.678540 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:24.678518 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-d9jxz" Aug 04 09:48:24.681179 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:24.681160 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mg4mx\"" Aug 04 09:48:24.696417 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:24.696394 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d9jxz"] Aug 04 09:48:24.854357 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:24.854329 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnz4b\" (UniqueName: \"kubernetes.io/projected/c04d0421-a0eb-475d-a371-de1b01571cb5-kube-api-access-jnz4b\") pod \"authorino-c548b5d7d-d9jxz\" (UID: \"c04d0421-a0eb-475d-a371-de1b01571cb5\") " pod="kuadrant-system/authorino-c548b5d7d-d9jxz" Aug 04 09:48:24.955099 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:24.955032 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnz4b\" (UniqueName: \"kubernetes.io/projected/c04d0421-a0eb-475d-a371-de1b01571cb5-kube-api-access-jnz4b\") pod \"authorino-c548b5d7d-d9jxz\" (UID: \"c04d0421-a0eb-475d-a371-de1b01571cb5\") " pod="kuadrant-system/authorino-c548b5d7d-d9jxz" Aug 04 09:48:24.965924 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:24.965901 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnz4b\" (UniqueName: \"kubernetes.io/projected/c04d0421-a0eb-475d-a371-de1b01571cb5-kube-api-access-jnz4b\") pod \"authorino-c548b5d7d-d9jxz\" (UID: \"c04d0421-a0eb-475d-a371-de1b01571cb5\") " pod="kuadrant-system/authorino-c548b5d7d-d9jxz" Aug 04 09:48:24.986784 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:24.986764 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-d9jxz" Aug 04 09:48:25.125879 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:25.125853 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d9jxz"] Aug 04 09:48:25.127827 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:48:25.127801 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc04d0421_a0eb_475d_a371_de1b01571cb5.slice/crio-abdf5e31de971a8fe0b61aad5aeb8d07881d901d0bcd0a5b1c75676910eb1d3b WatchSource:0}: Error finding container abdf5e31de971a8fe0b61aad5aeb8d07881d901d0bcd0a5b1c75676910eb1d3b: Status 404 returned error can't find the container with id abdf5e31de971a8fe0b61aad5aeb8d07881d901d0bcd0a5b1c75676910eb1d3b Aug 04 09:48:25.445841 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:25.445814 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-d9jxz" event={"ID":"c04d0421-a0eb-475d-a371-de1b01571cb5","Type":"ContainerStarted","Data":"abdf5e31de971a8fe0b61aad5aeb8d07881d901d0bcd0a5b1c75676910eb1d3b"} Aug 04 09:48:27.457411 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:27.457376 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-d9jxz" event={"ID":"c04d0421-a0eb-475d-a371-de1b01571cb5","Type":"ContainerStarted","Data":"55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46"} Aug 04 09:48:27.487550 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:27.487504 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-d9jxz" podStartSLOduration=1.472473295 podStartE2EDuration="3.487490625s" totalImagesPullingTime="2.01501733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:27.486931443 +0000 UTC m=+476.678223942" watchObservedRunningTime="2026-08-04 09:48:27.487490625 +0000 UTC m=+476.678783119" Aug 04 09:48:28.463661 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:28.463630 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" event={"ID":"f2a72770-b12d-42e8-9894-ad9b9f317c02","Type":"ContainerStarted","Data":"035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5"} Aug 04 09:48:28.464069 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:28.463779 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:28.489147 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:28.489106 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" podStartSLOduration=1.653077073 podStartE2EDuration="5.4890932s" totalImagesPullingTime="3.836016127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:28.488892232 +0000 UTC m=+477.680184706" watchObservedRunningTime="2026-08-04 09:48:28.4890932 +0000 UTC m=+477.680385695" Aug 04 09:48:37.855715 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:37.855677 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f2wwt"] Aug 04 09:48:37.856234 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:37.855894 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" podUID="f2a72770-b12d-42e8-9894-ad9b9f317c02" containerName="limitador" containerID="cri-o://035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5" gracePeriod=30 Aug 04 09:48:37.856579 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:37.856482 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:38.395408 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.395382 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:38.457594 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.457569 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f2a72770-b12d-42e8-9894-ad9b9f317c02-config-file\") pod \"f2a72770-b12d-42e8-9894-ad9b9f317c02\" (UID: \"f2a72770-b12d-42e8-9894-ad9b9f317c02\") " Aug 04 09:48:38.457789 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.457640 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbm8x\" (UniqueName: \"kubernetes.io/projected/f2a72770-b12d-42e8-9894-ad9b9f317c02-kube-api-access-sbm8x\") pod \"f2a72770-b12d-42e8-9894-ad9b9f317c02\" (UID: \"f2a72770-b12d-42e8-9894-ad9b9f317c02\") " Aug 04 09:48:38.457937 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.457914 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2a72770-b12d-42e8-9894-ad9b9f317c02-config-file" (OuterVolumeSpecName: "config-file") pod "f2a72770-b12d-42e8-9894-ad9b9f317c02" (UID: "f2a72770-b12d-42e8-9894-ad9b9f317c02"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:48:38.459369 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.459350 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2a72770-b12d-42e8-9894-ad9b9f317c02-kube-api-access-sbm8x" (OuterVolumeSpecName: "kube-api-access-sbm8x") pod "f2a72770-b12d-42e8-9894-ad9b9f317c02" (UID: "f2a72770-b12d-42e8-9894-ad9b9f317c02"). InnerVolumeSpecName "kube-api-access-sbm8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:48:38.507350 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.507320 2575 generic.go:356] "Generic (PLEG): container finished" podID="f2a72770-b12d-42e8-9894-ad9b9f317c02" containerID="035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5" exitCode=0 Aug 04 09:48:38.507447 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.507369 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" Aug 04 09:48:38.507447 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.507375 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" event={"ID":"f2a72770-b12d-42e8-9894-ad9b9f317c02","Type":"ContainerDied","Data":"035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5"} Aug 04 09:48:38.507447 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.507430 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-f2wwt" event={"ID":"f2a72770-b12d-42e8-9894-ad9b9f317c02","Type":"ContainerDied","Data":"11cca87eab4f7ef36645df3bf641650fc3f119f3d6b38efa689f81b571aa9398"} Aug 04 09:48:38.507552 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.507454 2575 scope.go:117] "RemoveContainer" containerID="035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5" Aug 04 09:48:38.515118 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.515100 2575 scope.go:117] "RemoveContainer" containerID="035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5" Aug 04 09:48:38.515398 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:48:38.515378 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5\": container with ID starting with 035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5 not found: ID does not exist" containerID="035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5" Aug 04 09:48:38.515472 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.515408 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5"} err="failed to get container status \"035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5\": rpc error: code = NotFound desc = could not find container \"035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5\": container with ID starting with 035c34b5c641832ec0e42b22c3faab8214a3e1637e934fcda4e7cdba16c31fd5 not found: ID does not exist" Aug 04 09:48:38.536763 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.536739 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f2wwt"] Aug 04 09:48:38.545674 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.545656 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f2wwt"] Aug 04 09:48:38.559046 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.559016 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbm8x\" (UniqueName: \"kubernetes.io/projected/f2a72770-b12d-42e8-9894-ad9b9f317c02-kube-api-access-sbm8x\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:48:38.559123 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:38.559048 2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f2a72770-b12d-42e8-9894-ad9b9f317c02-config-file\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:48:39.338119 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:39.338087 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2a72770-b12d-42e8-9894-ad9b9f317c02" path="/var/lib/kubelet/pods/f2a72770-b12d-42e8-9894-ad9b9f317c02/volumes" Aug 04 09:48:40.076265 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.076232 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd"] Aug 04 09:48:40.076578 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.076567 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2a72770-b12d-42e8-9894-ad9b9f317c02" containerName="limitador" Aug 04 09:48:40.076623 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.076579 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2a72770-b12d-42e8-9894-ad9b9f317c02" containerName="limitador" Aug 04 09:48:40.076655 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.076639 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2a72770-b12d-42e8-9894-ad9b9f317c02" containerName="limitador" Aug 04 09:48:40.079605 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.079589 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" Aug 04 09:48:40.083183 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.083158 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Aug 04 09:48:40.083290 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.083246 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zkt7v\"" Aug 04 09:48:40.084360 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.084345 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Aug 04 09:48:40.086223 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.086204 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 09:48:40.095327 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.095305 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd"] Aug 04 09:48:40.171878 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.171850 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlvdk\" (UniqueName: \"kubernetes.io/projected/f66e24bd-aa1e-4d93-9273-e2b35d254411-kube-api-access-hlvdk\") pod \"postgres-69fb4594cf-8cmvd\" (UID: \"f66e24bd-aa1e-4d93-9273-e2b35d254411\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" Aug 04 09:48:40.172040 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.171885 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f66e24bd-aa1e-4d93-9273-e2b35d254411-data\") pod \"postgres-69fb4594cf-8cmvd\" (UID: \"f66e24bd-aa1e-4d93-9273-e2b35d254411\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" Aug 04 09:48:40.272834 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.272802 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlvdk\" (UniqueName: \"kubernetes.io/projected/f66e24bd-aa1e-4d93-9273-e2b35d254411-kube-api-access-hlvdk\") pod \"postgres-69fb4594cf-8cmvd\" (UID: \"f66e24bd-aa1e-4d93-9273-e2b35d254411\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" Aug 04 09:48:40.272974 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.272840 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f66e24bd-aa1e-4d93-9273-e2b35d254411-data\") pod \"postgres-69fb4594cf-8cmvd\" (UID: \"f66e24bd-aa1e-4d93-9273-e2b35d254411\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" Aug 04 09:48:40.273169 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.273154 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f66e24bd-aa1e-4d93-9273-e2b35d254411-data\") pod \"postgres-69fb4594cf-8cmvd\" (UID: \"f66e24bd-aa1e-4d93-9273-e2b35d254411\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" Aug 04 09:48:40.286469 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.286443 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlvdk\" (UniqueName: \"kubernetes.io/projected/f66e24bd-aa1e-4d93-9273-e2b35d254411-kube-api-access-hlvdk\") pod \"postgres-69fb4594cf-8cmvd\" (UID: \"f66e24bd-aa1e-4d93-9273-e2b35d254411\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" Aug 04 09:48:40.389485 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.389431 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" Aug 04 09:48:40.538176 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:40.538146 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd"] Aug 04 09:48:40.541870 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:48:40.541843 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66e24bd_aa1e_4d93_9273_e2b35d254411.slice/crio-0d0bb16f0631fe59ac1e40f8da8083eb891ff8ba94456bc70b83efdc37103bd7 WatchSource:0}: Error finding container 0d0bb16f0631fe59ac1e40f8da8083eb891ff8ba94456bc70b83efdc37103bd7: Status 404 returned error can't find the container with id 0d0bb16f0631fe59ac1e40f8da8083eb891ff8ba94456bc70b83efdc37103bd7 Aug 04 09:48:41.527898 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:41.527857 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" event={"ID":"f66e24bd-aa1e-4d93-9273-e2b35d254411","Type":"ContainerStarted","Data":"0d0bb16f0631fe59ac1e40f8da8083eb891ff8ba94456bc70b83efdc37103bd7"} Aug 04 09:48:45.551475 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:45.551427 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" event={"ID":"f66e24bd-aa1e-4d93-9273-e2b35d254411","Type":"ContainerStarted","Data":"4d309eef4f6e1cdf241b2df2a53ddbafee1742b80d217041cb29fa3f655a1c6f"} Aug 04 09:48:45.551899 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:45.551523 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" Aug 04 09:48:45.570524 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:45.570425 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" podStartSLOduration=0.808408074 podStartE2EDuration="5.57040826s" totalImagesPullingTime="4.762000186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:45.569424583 +0000 UTC m=+494.760717116" watchObservedRunningTime="2026-08-04 09:48:45.57040826 +0000 UTC m=+494.761700756" Aug 04 09:48:51.583584 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:51.583559 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8cmvd" Aug 04 09:48:52.205793 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.205760 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-5gv4h"] Aug 04 09:48:52.211831 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.211813 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" Aug 04 09:48:52.219110 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.219091 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-5gv4h"] Aug 04 09:48:52.282523 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.282497 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf6dt\" (UniqueName: \"kubernetes.io/projected/e51b1644-e8dd-4f6e-a6f4-6c912780b630-kube-api-access-kf6dt\") pod \"authorino-5dc7ccb5f-5gv4h\" (UID: \"e51b1644-e8dd-4f6e-a6f4-6c912780b630\") " pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" Aug 04 09:48:52.383207 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.383178 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf6dt\" (UniqueName: \"kubernetes.io/projected/e51b1644-e8dd-4f6e-a6f4-6c912780b630-kube-api-access-kf6dt\") pod \"authorino-5dc7ccb5f-5gv4h\" (UID: \"e51b1644-e8dd-4f6e-a6f4-6c912780b630\") " pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" Aug 04 09:48:52.392774 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.392722 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf6dt\" (UniqueName: \"kubernetes.io/projected/e51b1644-e8dd-4f6e-a6f4-6c912780b630-kube-api-access-kf6dt\") pod \"authorino-5dc7ccb5f-5gv4h\" (UID: \"e51b1644-e8dd-4f6e-a6f4-6c912780b630\") " pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" Aug 04 09:48:52.481196 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.481134 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-5gv4h"] Aug 04 09:48:52.481355 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.481344 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" Aug 04 09:48:52.515336 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.515306 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64f97849d-dgqhp"] Aug 04 09:48:52.520968 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.520944 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64f97849d-dgqhp" Aug 04 09:48:52.528239 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.528217 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64f97849d-dgqhp"] Aug 04 09:48:52.584441 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.584416 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nslcn\" (UniqueName: \"kubernetes.io/projected/ce174967-42d5-4ca6-96d8-4cd29e1f9dbb-kube-api-access-nslcn\") pod \"authorino-64f97849d-dgqhp\" (UID: \"ce174967-42d5-4ca6-96d8-4cd29e1f9dbb\") " pod="kuadrant-system/authorino-64f97849d-dgqhp" Aug 04 09:48:52.604247 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.604220 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64f97849d-dgqhp"] Aug 04 09:48:52.604476 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:48:52.604458 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-nslcn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-64f97849d-dgqhp" podUID="ce174967-42d5-4ca6-96d8-4cd29e1f9dbb" Aug 04 09:48:52.621067 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.621046 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-5gv4h"] Aug 04 09:48:52.622887 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:48:52.622864 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode51b1644_e8dd_4f6e_a6f4_6c912780b630.slice/crio-d38ece45e5f97d3a24bde68672e72d6d70d6df557c85e75a7971d2419308b9b6 WatchSource:0}: Error finding container d38ece45e5f97d3a24bde68672e72d6d70d6df557c85e75a7971d2419308b9b6: Status 404 returned error can't find the container with id d38ece45e5f97d3a24bde68672e72d6d70d6df557c85e75a7971d2419308b9b6 Aug 04 09:48:52.643893 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.643869 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56d47f98bd-4blwv"] Aug 04 09:48:52.647915 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.647899 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d47f98bd-4blwv" Aug 04 09:48:52.650331 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.650306 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Aug 04 09:48:52.660833 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.660813 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56d47f98bd-4blwv"] Aug 04 09:48:52.685909 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.685885 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nslcn\" (UniqueName: \"kubernetes.io/projected/ce174967-42d5-4ca6-96d8-4cd29e1f9dbb-kube-api-access-nslcn\") pod \"authorino-64f97849d-dgqhp\" (UID: \"ce174967-42d5-4ca6-96d8-4cd29e1f9dbb\") " pod="kuadrant-system/authorino-64f97849d-dgqhp" Aug 04 09:48:52.695048 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.695029 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nslcn\" (UniqueName: \"kubernetes.io/projected/ce174967-42d5-4ca6-96d8-4cd29e1f9dbb-kube-api-access-nslcn\") pod \"authorino-64f97849d-dgqhp\" (UID: \"ce174967-42d5-4ca6-96d8-4cd29e1f9dbb\") " pod="kuadrant-system/authorino-64f97849d-dgqhp" Aug 04 09:48:52.786283 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.786227 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnpbj\" (UniqueName: \"kubernetes.io/projected/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-kube-api-access-tnpbj\") pod \"authorino-56d47f98bd-4blwv\" (UID: \"5e80e5ae-f64a-41df-9a86-7fe87d7ff543\") " pod="kuadrant-system/authorino-56d47f98bd-4blwv" Aug 04 09:48:52.786283 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.786259 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-tls-cert\") pod \"authorino-56d47f98bd-4blwv\" (UID: \"5e80e5ae-f64a-41df-9a86-7fe87d7ff543\") " pod="kuadrant-system/authorino-56d47f98bd-4blwv" Aug 04 09:48:52.887206 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.887176 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-tls-cert\") pod \"authorino-56d47f98bd-4blwv\" (UID: \"5e80e5ae-f64a-41df-9a86-7fe87d7ff543\") " pod="kuadrant-system/authorino-56d47f98bd-4blwv" Aug 04 09:48:52.887338 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.887322 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnpbj\" (UniqueName: \"kubernetes.io/projected/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-kube-api-access-tnpbj\") pod \"authorino-56d47f98bd-4blwv\" (UID: \"5e80e5ae-f64a-41df-9a86-7fe87d7ff543\") " pod="kuadrant-system/authorino-56d47f98bd-4blwv" Aug 04 09:48:52.889314 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.889295 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-tls-cert\") pod \"authorino-56d47f98bd-4blwv\" (UID: \"5e80e5ae-f64a-41df-9a86-7fe87d7ff543\") " pod="kuadrant-system/authorino-56d47f98bd-4blwv" Aug 04 09:48:52.896101 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.896078 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnpbj\" (UniqueName: \"kubernetes.io/projected/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-kube-api-access-tnpbj\") pod \"authorino-56d47f98bd-4blwv\" (UID: \"5e80e5ae-f64a-41df-9a86-7fe87d7ff543\") " pod="kuadrant-system/authorino-56d47f98bd-4blwv" Aug 04 09:48:52.957329 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:52.957308 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d47f98bd-4blwv" Aug 04 09:48:53.084846 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.084824 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56d47f98bd-4blwv"] Aug 04 09:48:53.087100 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:48:53.087076 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e80e5ae_f64a_41df_9a86_7fe87d7ff543.slice/crio-42c63c19b666d2c569af4935045471a8fb6680271ee885cf63442bbee3111799 WatchSource:0}: Error finding container 42c63c19b666d2c569af4935045471a8fb6680271ee885cf63442bbee3111799: Status 404 returned error can't find the container with id 42c63c19b666d2c569af4935045471a8fb6680271ee885cf63442bbee3111799 Aug 04 09:48:53.589122 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.589091 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d47f98bd-4blwv" event={"ID":"5e80e5ae-f64a-41df-9a86-7fe87d7ff543","Type":"ContainerStarted","Data":"42c63c19b666d2c569af4935045471a8fb6680271ee885cf63442bbee3111799"} Aug 04 09:48:53.590682 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.590651 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" event={"ID":"e51b1644-e8dd-4f6e-a6f4-6c912780b630","Type":"ContainerStarted","Data":"c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a"} Aug 04 09:48:53.590682 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.590674 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64f97849d-dgqhp" Aug 04 09:48:53.590837 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.590686 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" podUID="e51b1644-e8dd-4f6e-a6f4-6c912780b630" containerName="authorino" containerID="cri-o://c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a" gracePeriod=30 Aug 04 09:48:53.590837 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.590698 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" event={"ID":"e51b1644-e8dd-4f6e-a6f4-6c912780b630","Type":"ContainerStarted","Data":"d38ece45e5f97d3a24bde68672e72d6d70d6df557c85e75a7971d2419308b9b6"} Aug 04 09:48:53.599298 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.599280 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64f97849d-dgqhp" Aug 04 09:48:53.618776 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.618703 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" podStartSLOduration=1.122655629 podStartE2EDuration="1.618686612s" totalImagesPullingTime="496.030983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:53.616986452 +0000 UTC m=+502.808278948" watchObservedRunningTime="2026-08-04 09:48:53.618686612 +0000 UTC m=+502.809979107" Aug 04 09:48:53.693881 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.693854 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nslcn\" (UniqueName: \"kubernetes.io/projected/ce174967-42d5-4ca6-96d8-4cd29e1f9dbb-kube-api-access-nslcn\") pod \"ce174967-42d5-4ca6-96d8-4cd29e1f9dbb\" (UID: \"ce174967-42d5-4ca6-96d8-4cd29e1f9dbb\") " Aug 04 09:48:53.696132 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.696101 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce174967-42d5-4ca6-96d8-4cd29e1f9dbb-kube-api-access-nslcn" (OuterVolumeSpecName: "kube-api-access-nslcn") pod "ce174967-42d5-4ca6-96d8-4cd29e1f9dbb" (UID: "ce174967-42d5-4ca6-96d8-4cd29e1f9dbb"). InnerVolumeSpecName "kube-api-access-nslcn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:48:53.795205 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.795138 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nslcn\" (UniqueName: \"kubernetes.io/projected/ce174967-42d5-4ca6-96d8-4cd29e1f9dbb-kube-api-access-nslcn\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:48:53.864006 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.863978 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" Aug 04 09:48:53.996829 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.996800 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf6dt\" (UniqueName: \"kubernetes.io/projected/e51b1644-e8dd-4f6e-a6f4-6c912780b630-kube-api-access-kf6dt\") pod \"e51b1644-e8dd-4f6e-a6f4-6c912780b630\" (UID: \"e51b1644-e8dd-4f6e-a6f4-6c912780b630\") " Aug 04 09:48:53.998776 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:53.998745 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e51b1644-e8dd-4f6e-a6f4-6c912780b630-kube-api-access-kf6dt" (OuterVolumeSpecName: "kube-api-access-kf6dt") pod "e51b1644-e8dd-4f6e-a6f4-6c912780b630" (UID: "e51b1644-e8dd-4f6e-a6f4-6c912780b630"). InnerVolumeSpecName "kube-api-access-kf6dt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:48:54.098331 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.098266 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf6dt\" (UniqueName: \"kubernetes.io/projected/e51b1644-e8dd-4f6e-a6f4-6c912780b630-kube-api-access-kf6dt\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:48:54.597380 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.597343 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d47f98bd-4blwv" event={"ID":"5e80e5ae-f64a-41df-9a86-7fe87d7ff543","Type":"ContainerStarted","Data":"cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c"} Aug 04 09:48:54.598510 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.598488 2575 generic.go:356] "Generic (PLEG): container finished" podID="e51b1644-e8dd-4f6e-a6f4-6c912780b630" containerID="c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a" exitCode=0 Aug 04 09:48:54.598602 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.598524 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" Aug 04 09:48:54.598645 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.598520 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" event={"ID":"e51b1644-e8dd-4f6e-a6f4-6c912780b630","Type":"ContainerDied","Data":"c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a"} Aug 04 09:48:54.598645 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.598632 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-5gv4h" event={"ID":"e51b1644-e8dd-4f6e-a6f4-6c912780b630","Type":"ContainerDied","Data":"d38ece45e5f97d3a24bde68672e72d6d70d6df557c85e75a7971d2419308b9b6"} Aug 04 09:48:54.598755 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.598657 2575 scope.go:117] "RemoveContainer" containerID="c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a" Aug 04 09:48:54.598755 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.598702 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64f97849d-dgqhp" Aug 04 09:48:54.607630 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.607613 2575 scope.go:117] "RemoveContainer" containerID="c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a" Aug 04 09:48:54.607882 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:48:54.607860 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a\": container with ID starting with c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a not found: ID does not exist" containerID="c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a" Aug 04 09:48:54.607955 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.607888 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a"} err="failed to get container status \"c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a\": rpc error: code = NotFound desc = could not find container \"c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a\": container with ID starting with c4036daf564cbc69b6929beeca51f540bc7f6895b56c090a9583b9019e2fc23a not found: ID does not exist" Aug 04 09:48:54.613938 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.613873 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56d47f98bd-4blwv" podStartSLOduration=2.163430531 podStartE2EDuration="2.613859511s" totalImagesPullingTime="450.42898ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:54.613113271 +0000 UTC m=+503.804405766" watchObservedRunningTime="2026-08-04 09:48:54.613859511 +0000 UTC m=+503.805152005" Aug 04 09:48:54.640339 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.640308 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64f97849d-dgqhp"] Aug 04 09:48:54.645772 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.645747 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d9jxz"] Aug 04 09:48:54.645980 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.645954 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-d9jxz" podUID="c04d0421-a0eb-475d-a371-de1b01571cb5" containerName="authorino" containerID="cri-o://55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46" gracePeriod=30 Aug 04 09:48:54.650902 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.650845 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-64f97849d-dgqhp"] Aug 04 09:48:54.664258 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.664202 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-5gv4h"] Aug 04 09:48:54.666428 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.666404 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-5gv4h"] Aug 04 09:48:54.900145 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:54.900122 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-d9jxz" Aug 04 09:48:55.005241 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.005215 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnz4b\" (UniqueName: \"kubernetes.io/projected/c04d0421-a0eb-475d-a371-de1b01571cb5-kube-api-access-jnz4b\") pod \"c04d0421-a0eb-475d-a371-de1b01571cb5\" (UID: \"c04d0421-a0eb-475d-a371-de1b01571cb5\") " Aug 04 09:48:55.007165 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.007139 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c04d0421-a0eb-475d-a371-de1b01571cb5-kube-api-access-jnz4b" (OuterVolumeSpecName: "kube-api-access-jnz4b") pod "c04d0421-a0eb-475d-a371-de1b01571cb5" (UID: "c04d0421-a0eb-475d-a371-de1b01571cb5"). InnerVolumeSpecName "kube-api-access-jnz4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:48:55.105824 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.105797 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnz4b\" (UniqueName: \"kubernetes.io/projected/c04d0421-a0eb-475d-a371-de1b01571cb5-kube-api-access-jnz4b\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:48:55.340437 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.340356 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce174967-42d5-4ca6-96d8-4cd29e1f9dbb" path="/var/lib/kubelet/pods/ce174967-42d5-4ca6-96d8-4cd29e1f9dbb/volumes" Aug 04 09:48:55.340701 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.340679 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e51b1644-e8dd-4f6e-a6f4-6c912780b630" path="/var/lib/kubelet/pods/e51b1644-e8dd-4f6e-a6f4-6c912780b630/volumes" Aug 04 09:48:55.606402 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.606305 2575 generic.go:356] "Generic (PLEG): container finished" podID="c04d0421-a0eb-475d-a371-de1b01571cb5" containerID="55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46" exitCode=0 Aug 04 09:48:55.606402 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.606368 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-d9jxz" Aug 04 09:48:55.606402 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.606367 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-d9jxz" event={"ID":"c04d0421-a0eb-475d-a371-de1b01571cb5","Type":"ContainerDied","Data":"55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46"} Aug 04 09:48:55.607333 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.606429 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-d9jxz" event={"ID":"c04d0421-a0eb-475d-a371-de1b01571cb5","Type":"ContainerDied","Data":"abdf5e31de971a8fe0b61aad5aeb8d07881d901d0bcd0a5b1c75676910eb1d3b"} Aug 04 09:48:55.607333 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.606453 2575 scope.go:117] "RemoveContainer" containerID="55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46" Aug 04 09:48:55.618054 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.618028 2575 scope.go:117] "RemoveContainer" containerID="55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46" Aug 04 09:48:55.618395 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:48:55.618370 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46\": container with ID starting with 55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46 not found: ID does not exist" containerID="55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46" Aug 04 09:48:55.618492 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.618405 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46"} err="failed to get container status \"55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46\": rpc error: code = NotFound desc = could not find container \"55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46\": container with ID starting with 55b11d0fa1bc6c2a84256a996d91afb4989d699521a0fb5e52f3fe338f3ffb46 not found: ID does not exist" Aug 04 09:48:55.623870 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.623845 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d9jxz"] Aug 04 09:48:55.627201 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:55.627175 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d9jxz"] Aug 04 09:48:57.337930 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:57.337890 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c04d0421-a0eb-475d-a371-de1b01571cb5" path="/var/lib/kubelet/pods/c04d0421-a0eb-475d-a371-de1b01571cb5/volumes" Aug 04 09:48:59.751794 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.751761 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d4c55bb95-h944b"] Aug 04 09:48:59.752126 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.752080 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e51b1644-e8dd-4f6e-a6f4-6c912780b630" containerName="authorino" Aug 04 09:48:59.752126 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.752091 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e51b1644-e8dd-4f6e-a6f4-6c912780b630" containerName="authorino" Aug 04 09:48:59.752126 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.752114 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c04d0421-a0eb-475d-a371-de1b01571cb5" containerName="authorino" Aug 04 09:48:59.752126 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.752119 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c04d0421-a0eb-475d-a371-de1b01571cb5" containerName="authorino" Aug 04 09:48:59.752249 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.752175 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c04d0421-a0eb-475d-a371-de1b01571cb5" containerName="authorino" Aug 04 09:48:59.752249 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.752182 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e51b1644-e8dd-4f6e-a6f4-6c912780b630" containerName="authorino" Aug 04 09:48:59.756833 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.756815 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:48:59.758962 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.758928 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Aug 04 09:48:59.759113 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.759093 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jg7n5\"" Aug 04 09:48:59.759491 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.759473 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Aug 04 09:48:59.769467 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.769446 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d4c55bb95-h944b"] Aug 04 09:48:59.843939 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.843916 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/379b99a6-5b4e-4b02-9480-7b76aed7e218-webhook-cert\") pod \"maas-controller-d4c55bb95-h944b\" (UID: \"379b99a6-5b4e-4b02-9480-7b76aed7e218\") " pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:48:59.844026 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.843956 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnk8f\" (UniqueName: \"kubernetes.io/projected/379b99a6-5b4e-4b02-9480-7b76aed7e218-kube-api-access-hnk8f\") pod \"maas-controller-d4c55bb95-h944b\" (UID: \"379b99a6-5b4e-4b02-9480-7b76aed7e218\") " pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:48:59.945190 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.945164 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/379b99a6-5b4e-4b02-9480-7b76aed7e218-webhook-cert\") pod \"maas-controller-d4c55bb95-h944b\" (UID: \"379b99a6-5b4e-4b02-9480-7b76aed7e218\") " pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:48:59.945286 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.945203 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnk8f\" (UniqueName: \"kubernetes.io/projected/379b99a6-5b4e-4b02-9480-7b76aed7e218-kube-api-access-hnk8f\") pod \"maas-controller-d4c55bb95-h944b\" (UID: \"379b99a6-5b4e-4b02-9480-7b76aed7e218\") " pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:48:59.945341 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:48:59.945307 2575 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Aug 04 09:48:59.945377 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:48:59.945362 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379b99a6-5b4e-4b02-9480-7b76aed7e218-webhook-cert podName:379b99a6-5b4e-4b02-9480-7b76aed7e218 nodeName:}" failed. No retries permitted until 2026-08-04 09:49:00.445347869 +0000 UTC m=+509.636640342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/379b99a6-5b4e-4b02-9480-7b76aed7e218-webhook-cert") pod "maas-controller-d4c55bb95-h944b" (UID: "379b99a6-5b4e-4b02-9480-7b76aed7e218") : secret "maas-controller-webhook-cert" not found Aug 04 09:48:59.961553 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.961530 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnk8f\" (UniqueName: \"kubernetes.io/projected/379b99a6-5b4e-4b02-9480-7b76aed7e218-kube-api-access-hnk8f\") pod \"maas-controller-d4c55bb95-h944b\" (UID: \"379b99a6-5b4e-4b02-9480-7b76aed7e218\") " pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:48:59.967744 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.967711 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f954f7b5-x7qfs"] Aug 04 09:48:59.970301 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.970286 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:48:59.982376 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:48:59.982353 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-x7qfs"] Aug 04 09:49:00.045652 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.045585 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/078b003e-ce03-4c96-8dc6-e4e35b977cdb-webhook-cert\") pod \"maas-controller-7f954f7b5-x7qfs\" (UID: \"078b003e-ce03-4c96-8dc6-e4e35b977cdb\") " pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:49:00.045652 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.045643 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp9ms\" (UniqueName: \"kubernetes.io/projected/078b003e-ce03-4c96-8dc6-e4e35b977cdb-kube-api-access-bp9ms\") pod \"maas-controller-7f954f7b5-x7qfs\" (UID: \"078b003e-ce03-4c96-8dc6-e4e35b977cdb\") " pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:49:00.146696 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.146665 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/078b003e-ce03-4c96-8dc6-e4e35b977cdb-webhook-cert\") pod \"maas-controller-7f954f7b5-x7qfs\" (UID: \"078b003e-ce03-4c96-8dc6-e4e35b977cdb\") " pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:49:00.146867 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.146768 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp9ms\" (UniqueName: \"kubernetes.io/projected/078b003e-ce03-4c96-8dc6-e4e35b977cdb-kube-api-access-bp9ms\") pod \"maas-controller-7f954f7b5-x7qfs\" (UID: \"078b003e-ce03-4c96-8dc6-e4e35b977cdb\") " pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:49:00.146867 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:49:00.146830 2575 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Aug 04 09:49:00.146984 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:49:00.146883 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078b003e-ce03-4c96-8dc6-e4e35b977cdb-webhook-cert podName:078b003e-ce03-4c96-8dc6-e4e35b977cdb nodeName:}" failed. No retries permitted until 2026-08-04 09:49:00.646869103 +0000 UTC m=+509.838161576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/078b003e-ce03-4c96-8dc6-e4e35b977cdb-webhook-cert") pod "maas-controller-7f954f7b5-x7qfs" (UID: "078b003e-ce03-4c96-8dc6-e4e35b977cdb") : secret "maas-controller-webhook-cert" not found Aug 04 09:49:00.157907 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.157886 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp9ms\" (UniqueName: \"kubernetes.io/projected/078b003e-ce03-4c96-8dc6-e4e35b977cdb-kube-api-access-bp9ms\") pod \"maas-controller-7f954f7b5-x7qfs\" (UID: \"078b003e-ce03-4c96-8dc6-e4e35b977cdb\") " pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:49:00.449031 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.448999 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/379b99a6-5b4e-4b02-9480-7b76aed7e218-webhook-cert\") pod \"maas-controller-d4c55bb95-h944b\" (UID: \"379b99a6-5b4e-4b02-9480-7b76aed7e218\") " pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:49:00.451300 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.451271 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/379b99a6-5b4e-4b02-9480-7b76aed7e218-webhook-cert\") pod \"maas-controller-d4c55bb95-h944b\" (UID: \"379b99a6-5b4e-4b02-9480-7b76aed7e218\") " pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:49:00.651557 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.651501 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/078b003e-ce03-4c96-8dc6-e4e35b977cdb-webhook-cert\") pod \"maas-controller-7f954f7b5-x7qfs\" (UID: \"078b003e-ce03-4c96-8dc6-e4e35b977cdb\") " pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:49:00.653975 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.653952 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/078b003e-ce03-4c96-8dc6-e4e35b977cdb-webhook-cert\") pod \"maas-controller-7f954f7b5-x7qfs\" (UID: \"078b003e-ce03-4c96-8dc6-e4e35b977cdb\") " pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:49:00.666738 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.666702 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:49:00.817603 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.817577 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d4c55bb95-h944b"] Aug 04 09:49:00.819553 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:49:00.819528 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379b99a6_5b4e_4b02_9480_7b76aed7e218.slice/crio-9ac61afede3b3948725f89eb15053fbf8f14167bd62193ac92d9585727fb80c0 WatchSource:0}: Error finding container 9ac61afede3b3948725f89eb15053fbf8f14167bd62193ac92d9585727fb80c0: Status 404 returned error can't find the container with id 9ac61afede3b3948725f89eb15053fbf8f14167bd62193ac92d9585727fb80c0 Aug 04 09:49:00.879979 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:00.879946 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:49:01.027482 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:01.027454 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-x7qfs"] Aug 04 09:49:01.029267 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:49:01.029240 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078b003e_ce03_4c96_8dc6_e4e35b977cdb.slice/crio-d4504fce80cc29d3e35c2d56aad217e1d30a5c3c2d9c0b191bcbbf341b7d4b67 WatchSource:0}: Error finding container d4504fce80cc29d3e35c2d56aad217e1d30a5c3c2d9c0b191bcbbf341b7d4b67: Status 404 returned error can't find the container with id d4504fce80cc29d3e35c2d56aad217e1d30a5c3c2d9c0b191bcbbf341b7d4b67 Aug 04 09:49:01.646585 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:01.644000 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4c55bb95-h944b" event={"ID":"379b99a6-5b4e-4b02-9480-7b76aed7e218","Type":"ContainerStarted","Data":"9ac61afede3b3948725f89eb15053fbf8f14167bd62193ac92d9585727fb80c0"} Aug 04 09:49:01.650632 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:01.650572 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" event={"ID":"078b003e-ce03-4c96-8dc6-e4e35b977cdb","Type":"ContainerStarted","Data":"d4504fce80cc29d3e35c2d56aad217e1d30a5c3c2d9c0b191bcbbf341b7d4b67"} Aug 04 09:49:03.662035 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:03.661935 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" event={"ID":"078b003e-ce03-4c96-8dc6-e4e35b977cdb","Type":"ContainerStarted","Data":"47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b"} Aug 04 09:49:03.662035 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:03.662001 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:49:03.663454 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:03.663427 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4c55bb95-h944b" event={"ID":"379b99a6-5b4e-4b02-9480-7b76aed7e218","Type":"ContainerStarted","Data":"bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85"} Aug 04 09:49:03.663544 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:03.663485 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:49:03.702269 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:03.702209 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" podStartSLOduration=2.366078784 podStartE2EDuration="4.7021926s" totalImagesPullingTime="2.336113816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:49:03.69731805 +0000 UTC m=+512.888610545" watchObservedRunningTime="2026-08-04 09:49:03.7021926 +0000 UTC m=+512.893485073" Aug 04 09:49:03.730468 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:03.730406 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d4c55bb95-h944b" podStartSLOduration=2.177772395 podStartE2EDuration="4.730386387s" totalImagesPullingTime="2.552613992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:49:03.730344182 +0000 UTC m=+512.921636677" watchObservedRunningTime="2026-08-04 09:49:03.730386387 +0000 UTC m=+512.921678880" Aug 04 09:49:14.670270 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:14.670238 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:49:14.670657 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:14.670306 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:49:14.759148 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:14.759116 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d4c55bb95-h944b"] Aug 04 09:49:14.759310 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:14.759286 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-d4c55bb95-h944b" podUID="379b99a6-5b4e-4b02-9480-7b76aed7e218" containerName="manager" containerID="cri-o://bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85" gracePeriod=30 Aug 04 09:49:14.996091 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:14.996067 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:49:15.080759 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.080700 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/379b99a6-5b4e-4b02-9480-7b76aed7e218-webhook-cert\") pod \"379b99a6-5b4e-4b02-9480-7b76aed7e218\" (UID: \"379b99a6-5b4e-4b02-9480-7b76aed7e218\") " Aug 04 09:49:15.080924 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.080806 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnk8f\" (UniqueName: \"kubernetes.io/projected/379b99a6-5b4e-4b02-9480-7b76aed7e218-kube-api-access-hnk8f\") pod \"379b99a6-5b4e-4b02-9480-7b76aed7e218\" (UID: \"379b99a6-5b4e-4b02-9480-7b76aed7e218\") " Aug 04 09:49:15.082920 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.082890 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/379b99a6-5b4e-4b02-9480-7b76aed7e218-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "379b99a6-5b4e-4b02-9480-7b76aed7e218" (UID: "379b99a6-5b4e-4b02-9480-7b76aed7e218"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:49:15.083050 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.083023 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/379b99a6-5b4e-4b02-9480-7b76aed7e218-kube-api-access-hnk8f" (OuterVolumeSpecName: "kube-api-access-hnk8f") pod "379b99a6-5b4e-4b02-9480-7b76aed7e218" (UID: "379b99a6-5b4e-4b02-9480-7b76aed7e218"). InnerVolumeSpecName "kube-api-access-hnk8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:49:15.181780 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.181722 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/379b99a6-5b4e-4b02-9480-7b76aed7e218-webhook-cert\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:49:15.181780 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.181775 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnk8f\" (UniqueName: \"kubernetes.io/projected/379b99a6-5b4e-4b02-9480-7b76aed7e218-kube-api-access-hnk8f\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:49:15.723429 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.723397 2575 generic.go:356] "Generic (PLEG): container finished" podID="379b99a6-5b4e-4b02-9480-7b76aed7e218" containerID="bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85" exitCode=0 Aug 04 09:49:15.723838 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.723435 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4c55bb95-h944b" Aug 04 09:49:15.723838 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.723474 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4c55bb95-h944b" event={"ID":"379b99a6-5b4e-4b02-9480-7b76aed7e218","Type":"ContainerDied","Data":"bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85"} Aug 04 09:49:15.723838 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.723521 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4c55bb95-h944b" event={"ID":"379b99a6-5b4e-4b02-9480-7b76aed7e218","Type":"ContainerDied","Data":"9ac61afede3b3948725f89eb15053fbf8f14167bd62193ac92d9585727fb80c0"} Aug 04 09:49:15.723838 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.723544 2575 scope.go:117] "RemoveContainer" containerID="bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85" Aug 04 09:49:15.731557 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.731541 2575 scope.go:117] "RemoveContainer" containerID="bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85" Aug 04 09:49:15.731819 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:49:15.731799 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85\": container with ID starting with bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85 not found: ID does not exist" containerID="bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85" Aug 04 09:49:15.731882 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.731827 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85"} err="failed to get container status \"bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85\": rpc error: code = NotFound desc = could not find container \"bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85\": container with ID starting with bee02ee7b52c93f69f368c836c64449997c5842ea80c3d03c88897de05af4b85 not found: ID does not exist" Aug 04 09:49:15.742515 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.742488 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d4c55bb95-h944b"] Aug 04 09:49:15.744295 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:15.744275 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d4c55bb95-h944b"] Aug 04 09:49:17.338789 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:17.338757 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="379b99a6-5b4e-4b02-9480-7b76aed7e218" path="/var/lib/kubelet/pods/379b99a6-5b4e-4b02-9480-7b76aed7e218/volumes" Aug 04 09:49:37.380231 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.380192 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh"] Aug 04 09:49:37.380723 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.380542 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="379b99a6-5b4e-4b02-9480-7b76aed7e218" containerName="manager" Aug 04 09:49:37.380723 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.380553 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="379b99a6-5b4e-4b02-9480-7b76aed7e218" containerName="manager" Aug 04 09:49:37.380723 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.380622 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="379b99a6-5b4e-4b02-9480-7b76aed7e218" containerName="manager" Aug 04 09:49:37.387857 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.387835 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:37.395535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.395516 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Aug 04 09:49:37.395643 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.395516 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9sr5j\"" Aug 04 09:49:37.412650 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.412626 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh"] Aug 04 09:49:37.477544 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.477510 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e983e235-7b65-40e8-86d3-65b753ae5209-openshift-service-ca\") pod \"maas-api-6bc7d6b7f8-hkgnh\" (UID: \"e983e235-7b65-40e8-86d3-65b753ae5209\") " pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:37.477704 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.477573 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkskd\" (UniqueName: \"kubernetes.io/projected/e983e235-7b65-40e8-86d3-65b753ae5209-kube-api-access-zkskd\") pod \"maas-api-6bc7d6b7f8-hkgnh\" (UID: \"e983e235-7b65-40e8-86d3-65b753ae5209\") " pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:37.477704 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.477615 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e983e235-7b65-40e8-86d3-65b753ae5209-maas-api-tls\") pod \"maas-api-6bc7d6b7f8-hkgnh\" (UID: \"e983e235-7b65-40e8-86d3-65b753ae5209\") " pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:37.578984 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.578905 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e983e235-7b65-40e8-86d3-65b753ae5209-openshift-service-ca\") pod \"maas-api-6bc7d6b7f8-hkgnh\" (UID: \"e983e235-7b65-40e8-86d3-65b753ae5209\") " pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:37.579136 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.579034 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkskd\" (UniqueName: \"kubernetes.io/projected/e983e235-7b65-40e8-86d3-65b753ae5209-kube-api-access-zkskd\") pod \"maas-api-6bc7d6b7f8-hkgnh\" (UID: \"e983e235-7b65-40e8-86d3-65b753ae5209\") " pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:37.579136 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.579074 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e983e235-7b65-40e8-86d3-65b753ae5209-maas-api-tls\") pod \"maas-api-6bc7d6b7f8-hkgnh\" (UID: \"e983e235-7b65-40e8-86d3-65b753ae5209\") " pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:37.579584 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.579557 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e983e235-7b65-40e8-86d3-65b753ae5209-openshift-service-ca\") pod \"maas-api-6bc7d6b7f8-hkgnh\" (UID: \"e983e235-7b65-40e8-86d3-65b753ae5209\") " pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:37.581686 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.581661 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e983e235-7b65-40e8-86d3-65b753ae5209-maas-api-tls\") pod \"maas-api-6bc7d6b7f8-hkgnh\" (UID: \"e983e235-7b65-40e8-86d3-65b753ae5209\") " pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:37.592246 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.592220 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkskd\" (UniqueName: \"kubernetes.io/projected/e983e235-7b65-40e8-86d3-65b753ae5209-kube-api-access-zkskd\") pod \"maas-api-6bc7d6b7f8-hkgnh\" (UID: \"e983e235-7b65-40e8-86d3-65b753ae5209\") " pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:37.698623 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.698595 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:37.833958 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:37.833926 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh"] Aug 04 09:49:37.837066 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:49:37.837033 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode983e235_7b65_40e8_86d3_65b753ae5209.slice/crio-11bb0a146c2c5e3e60137afe8789884a39d3231ee6de43e04ffdeeb54660cb4e WatchSource:0}: Error finding container 11bb0a146c2c5e3e60137afe8789884a39d3231ee6de43e04ffdeeb54660cb4e: Status 404 returned error can't find the container with id 11bb0a146c2c5e3e60137afe8789884a39d3231ee6de43e04ffdeeb54660cb4e Aug 04 09:49:38.194435 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.194403 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh"] Aug 04 09:49:38.200484 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.200468 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" Aug 04 09:49:38.204535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.204517 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Aug 04 09:49:38.204535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.204531 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-sgk5v\"" Aug 04 09:49:38.216530 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.216508 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh"] Aug 04 09:49:38.285106 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.285074 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/645e6a6d-0310-4e8b-9e62-8d8a29ad6486-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4v2jh\" (UID: \"645e6a6d-0310-4e8b-9e62-8d8a29ad6486\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" Aug 04 09:49:38.285273 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.285135 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjjj7\" (UniqueName: \"kubernetes.io/projected/645e6a6d-0310-4e8b-9e62-8d8a29ad6486-kube-api-access-cjjj7\") pod \"payload-pre-processing-687dbb6cf6-4v2jh\" (UID: \"645e6a6d-0310-4e8b-9e62-8d8a29ad6486\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" Aug 04 09:49:38.385826 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.385795 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/645e6a6d-0310-4e8b-9e62-8d8a29ad6486-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4v2jh\" (UID: \"645e6a6d-0310-4e8b-9e62-8d8a29ad6486\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" Aug 04 09:49:38.386175 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.385890 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjjj7\" (UniqueName: \"kubernetes.io/projected/645e6a6d-0310-4e8b-9e62-8d8a29ad6486-kube-api-access-cjjj7\") pod \"payload-pre-processing-687dbb6cf6-4v2jh\" (UID: \"645e6a6d-0310-4e8b-9e62-8d8a29ad6486\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" Aug 04 09:49:38.386477 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.386459 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/645e6a6d-0310-4e8b-9e62-8d8a29ad6486-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4v2jh\" (UID: \"645e6a6d-0310-4e8b-9e62-8d8a29ad6486\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" Aug 04 09:49:38.400071 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.400040 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjjj7\" (UniqueName: \"kubernetes.io/projected/645e6a6d-0310-4e8b-9e62-8d8a29ad6486-kube-api-access-cjjj7\") pod \"payload-pre-processing-687dbb6cf6-4v2jh\" (UID: \"645e6a6d-0310-4e8b-9e62-8d8a29ad6486\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" Aug 04 09:49:38.511365 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.511291 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" Aug 04 09:49:38.836023 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.835942 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" event={"ID":"e983e235-7b65-40e8-86d3-65b753ae5209","Type":"ContainerStarted","Data":"11bb0a146c2c5e3e60137afe8789884a39d3231ee6de43e04ffdeeb54660cb4e"} Aug 04 09:49:38.863247 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:38.863217 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh"] Aug 04 09:49:38.865759 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:49:38.865715 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod645e6a6d_0310_4e8b_9e62_8d8a29ad6486.slice/crio-18531fa64867f051577d8299322d9ac2a2f2894617eb4d7867796534464c1f57 WatchSource:0}: Error finding container 18531fa64867f051577d8299322d9ac2a2f2894617eb4d7867796534464c1f57: Status 404 returned error can't find the container with id 18531fa64867f051577d8299322d9ac2a2f2894617eb4d7867796534464c1f57 Aug 04 09:49:39.844880 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:39.844705 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" event={"ID":"e983e235-7b65-40e8-86d3-65b753ae5209","Type":"ContainerStarted","Data":"17563b37a78d0743644c105eca091eda73f04088d9ad78dddd03792ce5033ad4"} Aug 04 09:49:39.845521 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:39.845474 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:39.846594 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:39.846532 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" event={"ID":"645e6a6d-0310-4e8b-9e62-8d8a29ad6486","Type":"ContainerStarted","Data":"18531fa64867f051577d8299322d9ac2a2f2894617eb4d7867796534464c1f57"} Aug 04 09:49:39.867772 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:39.867695 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" podStartSLOduration=1.136007343 podStartE2EDuration="2.867679102s" totalImagesPullingTime="1.731671759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:49:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:49:39.866994885 +0000 UTC m=+549.058287397" watchObservedRunningTime="2026-08-04 09:49:39.867679102 +0000 UTC m=+549.058971598" Aug 04 09:49:41.857833 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:41.857718 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" event={"ID":"645e6a6d-0310-4e8b-9e62-8d8a29ad6486","Type":"ContainerStarted","Data":"5a9459bd401562719b7fb2c7cb1818e139ffed781d227f637164d3d35c04cd15"} Aug 04 09:49:41.858275 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:41.857841 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" Aug 04 09:49:41.880531 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:41.880481 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" podStartSLOduration=1.284441537 podStartE2EDuration="3.880466591s" totalImagesPullingTime="2.596025054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:49:41.880118641 +0000 UTC m=+551.071411135" watchObservedRunningTime="2026-08-04 09:49:41.880466591 +0000 UTC m=+551.071759087" Aug 04 09:49:46.862885 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:46.862858 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6bc7d6b7f8-hkgnh" Aug 04 09:49:51.766035 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.766006 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg"] Aug 04 09:49:51.769687 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.769670 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:51.772197 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.772174 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 04 09:49:51.772283 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.772195 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Aug 04 09:49:51.772283 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.772197 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 04 09:49:51.772391 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.772263 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z7ff4\"" Aug 04 09:49:51.782088 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.782070 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg"] Aug 04 09:49:51.901087 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.901061 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:51.901247 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.901094 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:51.901247 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.901168 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4g6h\" (UniqueName: \"kubernetes.io/projected/c6af1930-27a1-4883-995a-390a96400f62-kube-api-access-v4g6h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:51.901247 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.901218 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:51.901247 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.901242 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6af1930-27a1-4883-995a-390a96400f62-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:51.901420 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:51.901281 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.001877 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.001830 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.001877 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.001876 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.002057 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.001901 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.002057 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.001949 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4g6h\" (UniqueName: \"kubernetes.io/projected/c6af1930-27a1-4883-995a-390a96400f62-kube-api-access-v4g6h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.002057 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.001985 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.002057 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.002013 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6af1930-27a1-4883-995a-390a96400f62-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.002287 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.002263 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.002406 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.002385 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.002508 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.002491 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.004300 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.004282 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6af1930-27a1-4883-995a-390a96400f62-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.004417 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.004400 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6af1930-27a1-4883-995a-390a96400f62-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.012980 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.012958 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4g6h\" (UniqueName: \"kubernetes.io/projected/c6af1930-27a1-4883-995a-390a96400f62-kube-api-access-v4g6h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg\" (UID: \"c6af1930-27a1-4883-995a-390a96400f62\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.080348 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.080304 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:49:52.213309 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.213283 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg"] Aug 04 09:49:52.215192 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:49:52.215167 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6af1930_27a1_4883_995a_390a96400f62.slice/crio-a999b2b59d00b71cbfe0a09cad60032c6b5018d273bc5aa46b1cd654d34f5390 WatchSource:0}: Error finding container a999b2b59d00b71cbfe0a09cad60032c6b5018d273bc5aa46b1cd654d34f5390: Status 404 returned error can't find the container with id a999b2b59d00b71cbfe0a09cad60032c6b5018d273bc5aa46b1cd654d34f5390 Aug 04 09:49:52.865845 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.865813 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4v2jh" Aug 04 09:49:52.910910 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:52.910879 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" event={"ID":"c6af1930-27a1-4883-995a-390a96400f62","Type":"ContainerStarted","Data":"a999b2b59d00b71cbfe0a09cad60032c6b5018d273bc5aa46b1cd654d34f5390"} Aug 04 09:49:53.916774 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:53.916720 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" event={"ID":"c6af1930-27a1-4883-995a-390a96400f62","Type":"ContainerStarted","Data":"9a69bb5a4453f6a545cebaf83596d9edb517b4363e240d8290f5205c30bf54bf"} Aug 04 09:49:53.940267 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:49:53.940214 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" podStartSLOduration=1.7696292850000002 podStartE2EDuration="2.940199732s" totalImagesPullingTime="1.170570447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:49:53.938678556 +0000 UTC m=+563.129971051" watchObservedRunningTime="2026-08-04 09:49:53.940199732 +0000 UTC m=+563.131492256" Aug 04 09:50:02.081175 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:02.081129 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:50:02.081691 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:02.081212 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:50:02.093416 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:02.093386 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:50:02.967706 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:02.967674 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg" Aug 04 09:50:15.419407 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.419376 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-272zf"] Aug 04 09:50:15.423920 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.423899 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.427250 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.427232 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Aug 04 09:50:15.439770 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.439747 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-272zf"] Aug 04 09:50:15.511103 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.511079 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.511211 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.511124 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.511211 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.511191 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.511323 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.511247 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.511323 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.511307 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8117de6f-e290-4954-8da1-370386428f85-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.511439 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.511355 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29p74\" (UniqueName: \"kubernetes.io/projected/8117de6f-e290-4954-8da1-370386428f85-kube-api-access-29p74\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.611703 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.611675 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.611873 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.611741 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.611873 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.611783 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.611873 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.611810 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.611873 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.611858 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8117de6f-e290-4954-8da1-370386428f85-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.612070 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.611905 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29p74\" (UniqueName: \"kubernetes.io/projected/8117de6f-e290-4954-8da1-370386428f85-kube-api-access-29p74\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.612138 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.612106 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.612192 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.612142 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.612307 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.612286 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.613978 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.613959 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8117de6f-e290-4954-8da1-370386428f85-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.614147 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.614130 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8117de6f-e290-4954-8da1-370386428f85-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.620809 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.620785 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29p74\" (UniqueName: \"kubernetes.io/projected/8117de6f-e290-4954-8da1-370386428f85-kube-api-access-29p74\") pod \"e2e-trlp-test-simulated-kserve-598466548d-272zf\" (UID: \"8117de6f-e290-4954-8da1-370386428f85\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.734558 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.734533 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:15.862525 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:15.862497 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-272zf"] Aug 04 09:50:15.865057 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:50:15.865035 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8117de6f_e290_4954_8da1_370386428f85.slice/crio-5510faa0fd564d2e0c5e7d1d17ddcee12d952e4808b0033a8c366adb3d47f3c7 WatchSource:0}: Error finding container 5510faa0fd564d2e0c5e7d1d17ddcee12d952e4808b0033a8c366adb3d47f3c7: Status 404 returned error can't find the container with id 5510faa0fd564d2e0c5e7d1d17ddcee12d952e4808b0033a8c366adb3d47f3c7 Aug 04 09:50:16.015826 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:16.015753 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" event={"ID":"8117de6f-e290-4954-8da1-370386428f85","Type":"ContainerStarted","Data":"5510faa0fd564d2e0c5e7d1d17ddcee12d952e4808b0033a8c366adb3d47f3c7"} Aug 04 09:50:17.022781 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:17.022738 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" event={"ID":"8117de6f-e290-4954-8da1-370386428f85","Type":"ContainerStarted","Data":"d02f9f88c08cfa3c80193efa6c7456efbb87c4d6eb6c6a9e16ee3cd937662330"} Aug 04 09:50:17.116657 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:17.116600 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" podStartSLOduration=1.6459707749999999 podStartE2EDuration="2.116583116s" totalImagesPullingTime="470.612341ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:50:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:50:17.11529019 +0000 UTC m=+586.306582686" watchObservedRunningTime="2026-08-04 09:50:17.116583116 +0000 UTC m=+586.307875610" Aug 04 09:50:25.735063 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:25.735018 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:25.735456 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:25.735088 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:25.747808 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:25.747780 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:26.079270 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:26.079187 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-272zf" Aug 04 09:50:31.197438 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:31.197412 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 09:50:31.208366 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:50:31.208344 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 09:51:47.033659 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.033616 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-584647fd45-87rrn"] Aug 04 09:51:47.037443 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.037420 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-584647fd45-87rrn" Aug 04 09:51:47.087771 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.087744 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-584647fd45-87rrn"] Aug 04 09:51:47.194246 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.194217 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwgg9\" (UniqueName: \"kubernetes.io/projected/e6c93048-1081-4ce6-a996-92398a7952af-kube-api-access-qwgg9\") pod \"authorino-584647fd45-87rrn\" (UID: \"e6c93048-1081-4ce6-a996-92398a7952af\") " pod="kuadrant-system/authorino-584647fd45-87rrn" Aug 04 09:51:47.194350 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.194269 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e6c93048-1081-4ce6-a996-92398a7952af-tls-cert\") pod \"authorino-584647fd45-87rrn\" (UID: \"e6c93048-1081-4ce6-a996-92398a7952af\") " pod="kuadrant-system/authorino-584647fd45-87rrn" Aug 04 09:51:47.295431 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.295372 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e6c93048-1081-4ce6-a996-92398a7952af-tls-cert\") pod \"authorino-584647fd45-87rrn\" (UID: \"e6c93048-1081-4ce6-a996-92398a7952af\") " pod="kuadrant-system/authorino-584647fd45-87rrn" Aug 04 09:51:47.295543 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.295471 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwgg9\" (UniqueName: \"kubernetes.io/projected/e6c93048-1081-4ce6-a996-92398a7952af-kube-api-access-qwgg9\") pod \"authorino-584647fd45-87rrn\" (UID: \"e6c93048-1081-4ce6-a996-92398a7952af\") " pod="kuadrant-system/authorino-584647fd45-87rrn" Aug 04 09:51:47.297644 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.297618 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e6c93048-1081-4ce6-a996-92398a7952af-tls-cert\") pod \"authorino-584647fd45-87rrn\" (UID: \"e6c93048-1081-4ce6-a996-92398a7952af\") " pod="kuadrant-system/authorino-584647fd45-87rrn" Aug 04 09:51:47.305185 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.305162 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwgg9\" (UniqueName: \"kubernetes.io/projected/e6c93048-1081-4ce6-a996-92398a7952af-kube-api-access-qwgg9\") pod \"authorino-584647fd45-87rrn\" (UID: \"e6c93048-1081-4ce6-a996-92398a7952af\") " pod="kuadrant-system/authorino-584647fd45-87rrn" Aug 04 09:51:47.346763 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.346742 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-584647fd45-87rrn" Aug 04 09:51:47.490099 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.490075 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-584647fd45-87rrn"] Aug 04 09:51:47.491495 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:51:47.491467 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6c93048_1081_4ce6_a996_92398a7952af.slice/crio-206020bfbe6230b385b2a5071e569eb2ff9efa840d4dd3544fb18b8e1e887a2d WatchSource:0}: Error finding container 206020bfbe6230b385b2a5071e569eb2ff9efa840d4dd3544fb18b8e1e887a2d: Status 404 returned error can't find the container with id 206020bfbe6230b385b2a5071e569eb2ff9efa840d4dd3544fb18b8e1e887a2d Aug 04 09:51:47.492688 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:47.492672 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 09:51:48.470038 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:48.469964 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-584647fd45-87rrn" event={"ID":"e6c93048-1081-4ce6-a996-92398a7952af","Type":"ContainerStarted","Data":"0048bbb7dd1efdd3e009de5466bac00025761e44b2411e985044b8de926572bb"} Aug 04 09:51:48.470038 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:48.470001 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-584647fd45-87rrn" event={"ID":"e6c93048-1081-4ce6-a996-92398a7952af","Type":"ContainerStarted","Data":"206020bfbe6230b385b2a5071e569eb2ff9efa840d4dd3544fb18b8e1e887a2d"} Aug 04 09:51:48.510948 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:48.510904 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-584647fd45-87rrn" podStartSLOduration=1.803355658 podStartE2EDuration="2.510889127s" totalImagesPullingTime="707.533469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:51:48.506323907 +0000 UTC m=+677.697616401" watchObservedRunningTime="2026-08-04 09:51:48.510889127 +0000 UTC m=+677.702181663" Aug 04 09:51:48.557045 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:48.556941 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56d47f98bd-4blwv"] Aug 04 09:51:48.557231 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:48.557179 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-56d47f98bd-4blwv" podUID="5e80e5ae-f64a-41df-9a86-7fe87d7ff543" containerName="authorino" containerID="cri-o://cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c" gracePeriod=30 Aug 04 09:51:48.810835 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:48.810803 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d47f98bd-4blwv" Aug 04 09:51:48.909272 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:48.909238 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnpbj\" (UniqueName: \"kubernetes.io/projected/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-kube-api-access-tnpbj\") pod \"5e80e5ae-f64a-41df-9a86-7fe87d7ff543\" (UID: \"5e80e5ae-f64a-41df-9a86-7fe87d7ff543\") " Aug 04 09:51:48.909427 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:48.909319 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-tls-cert\") pod \"5e80e5ae-f64a-41df-9a86-7fe87d7ff543\" (UID: \"5e80e5ae-f64a-41df-9a86-7fe87d7ff543\") " Aug 04 09:51:48.911408 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:48.911374 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-kube-api-access-tnpbj" (OuterVolumeSpecName: "kube-api-access-tnpbj") pod "5e80e5ae-f64a-41df-9a86-7fe87d7ff543" (UID: "5e80e5ae-f64a-41df-9a86-7fe87d7ff543"). InnerVolumeSpecName "kube-api-access-tnpbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:51:48.921129 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:48.921105 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5e80e5ae-f64a-41df-9a86-7fe87d7ff543" (UID: "5e80e5ae-f64a-41df-9a86-7fe87d7ff543"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:51:49.010189 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:49.010093 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnpbj\" (UniqueName: \"kubernetes.io/projected/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-kube-api-access-tnpbj\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:51:49.010189 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:49.010121 2575 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5e80e5ae-f64a-41df-9a86-7fe87d7ff543-tls-cert\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:51:49.477403 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:49.477359 2575 generic.go:356] "Generic (PLEG): container finished" podID="5e80e5ae-f64a-41df-9a86-7fe87d7ff543" containerID="cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c" exitCode=0 Aug 04 09:51:49.477874 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:49.477406 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d47f98bd-4blwv" event={"ID":"5e80e5ae-f64a-41df-9a86-7fe87d7ff543","Type":"ContainerDied","Data":"cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c"} Aug 04 09:51:49.477874 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:49.477469 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d47f98bd-4blwv" event={"ID":"5e80e5ae-f64a-41df-9a86-7fe87d7ff543","Type":"ContainerDied","Data":"42c63c19b666d2c569af4935045471a8fb6680271ee885cf63442bbee3111799"} Aug 04 09:51:49.477874 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:49.477503 2575 scope.go:117] "RemoveContainer" containerID="cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c" Aug 04 09:51:49.477874 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:49.477632 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d47f98bd-4blwv" Aug 04 09:51:49.487435 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:49.487415 2575 scope.go:117] "RemoveContainer" containerID="cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c" Aug 04 09:51:49.487679 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:51:49.487661 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c\": container with ID starting with cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c not found: ID does not exist" containerID="cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c" Aug 04 09:51:49.487776 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:49.487687 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c"} err="failed to get container status \"cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c\": rpc error: code = NotFound desc = could not find container \"cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c\": container with ID starting with cb8ef915f65f6119d5da9ee8611da6c88cab860bee0f966d61da6aa3b437b16c not found: ID does not exist" Aug 04 09:51:49.517920 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:49.517896 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56d47f98bd-4blwv"] Aug 04 09:51:49.539469 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:49.539443 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56d47f98bd-4blwv"] Aug 04 09:51:51.341076 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:51:51.341037 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e80e5ae-f64a-41df-9a86-7fe87d7ff543" path="/var/lib/kubelet/pods/5e80e5ae-f64a-41df-9a86-7fe87d7ff543/volumes" Aug 04 09:52:40.094205 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.094170 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-x7qfs"] Aug 04 09:52:40.094625 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.094410 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" podUID="078b003e-ce03-4c96-8dc6-e4e35b977cdb" containerName="manager" containerID="cri-o://47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b" gracePeriod=30 Aug 04 09:52:40.340529 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.340505 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:52:40.424671 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.424608 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/078b003e-ce03-4c96-8dc6-e4e35b977cdb-webhook-cert\") pod \"078b003e-ce03-4c96-8dc6-e4e35b977cdb\" (UID: \"078b003e-ce03-4c96-8dc6-e4e35b977cdb\") " Aug 04 09:52:40.424852 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.424680 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp9ms\" (UniqueName: \"kubernetes.io/projected/078b003e-ce03-4c96-8dc6-e4e35b977cdb-kube-api-access-bp9ms\") pod \"078b003e-ce03-4c96-8dc6-e4e35b977cdb\" (UID: \"078b003e-ce03-4c96-8dc6-e4e35b977cdb\") " Aug 04 09:52:40.426452 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.426424 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/078b003e-ce03-4c96-8dc6-e4e35b977cdb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "078b003e-ce03-4c96-8dc6-e4e35b977cdb" (UID: "078b003e-ce03-4c96-8dc6-e4e35b977cdb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:52:40.426571 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.426549 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/078b003e-ce03-4c96-8dc6-e4e35b977cdb-kube-api-access-bp9ms" (OuterVolumeSpecName: "kube-api-access-bp9ms") pod "078b003e-ce03-4c96-8dc6-e4e35b977cdb" (UID: "078b003e-ce03-4c96-8dc6-e4e35b977cdb"). InnerVolumeSpecName "kube-api-access-bp9ms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:52:40.525925 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.525904 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/078b003e-ce03-4c96-8dc6-e4e35b977cdb-webhook-cert\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:52:40.525925 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.525927 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp9ms\" (UniqueName: \"kubernetes.io/projected/078b003e-ce03-4c96-8dc6-e4e35b977cdb-kube-api-access-bp9ms\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:52:40.731429 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.731399 2575 generic.go:356] "Generic (PLEG): container finished" podID="078b003e-ce03-4c96-8dc6-e4e35b977cdb" containerID="47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b" exitCode=0 Aug 04 09:52:40.731535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.731436 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" Aug 04 09:52:40.731535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.731448 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" event={"ID":"078b003e-ce03-4c96-8dc6-e4e35b977cdb","Type":"ContainerDied","Data":"47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b"} Aug 04 09:52:40.731535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.731495 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f954f7b5-x7qfs" event={"ID":"078b003e-ce03-4c96-8dc6-e4e35b977cdb","Type":"ContainerDied","Data":"d4504fce80cc29d3e35c2d56aad217e1d30a5c3c2d9c0b191bcbbf341b7d4b67"} Aug 04 09:52:40.731535 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.731525 2575 scope.go:117] "RemoveContainer" containerID="47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b" Aug 04 09:52:40.740001 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.739983 2575 scope.go:117] "RemoveContainer" containerID="47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b" Aug 04 09:52:40.740256 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:52:40.740236 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b\": container with ID starting with 47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b not found: ID does not exist" containerID="47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b" Aug 04 09:52:40.740302 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.740265 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b"} err="failed to get container status \"47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b\": rpc error: code = NotFound desc = could not find container \"47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b\": container with ID starting with 47c1be5463495e4f5003e273460f66077931ac9a1b04cf7a4760462b29d5103b not found: ID does not exist" Aug 04 09:52:40.763068 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.763047 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-x7qfs"] Aug 04 09:52:40.768158 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:40.768140 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-x7qfs"] Aug 04 09:52:41.339487 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:52:41.339453 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="078b003e-ce03-4c96-8dc6-e4e35b977cdb" path="/var/lib/kubelet/pods/078b003e-ce03-4c96-8dc6-e4e35b977cdb/volumes" Aug 04 09:53:41.748441 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.748401 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f954f7b5-z94c4"] Aug 04 09:53:41.748980 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.748841 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e80e5ae-f64a-41df-9a86-7fe87d7ff543" containerName="authorino" Aug 04 09:53:41.748980 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.748855 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e80e5ae-f64a-41df-9a86-7fe87d7ff543" containerName="authorino" Aug 04 09:53:41.748980 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.748865 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="078b003e-ce03-4c96-8dc6-e4e35b977cdb" containerName="manager" Aug 04 09:53:41.748980 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.748871 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="078b003e-ce03-4c96-8dc6-e4e35b977cdb" containerName="manager" Aug 04 09:53:41.748980 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.748959 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="078b003e-ce03-4c96-8dc6-e4e35b977cdb" containerName="manager" Aug 04 09:53:41.748980 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.748972 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e80e5ae-f64a-41df-9a86-7fe87d7ff543" containerName="authorino" Aug 04 09:53:41.753324 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.753302 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:41.758656 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.758632 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jg7n5\"" Aug 04 09:53:41.758969 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.758950 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Aug 04 09:53:41.759081 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.758994 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Aug 04 09:53:41.780928 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.780898 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-z94c4"] Aug 04 09:53:41.783649 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.783594 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m5wz\" (UniqueName: \"kubernetes.io/projected/c01cada3-db6d-4757-8a5c-96c6296d3088-kube-api-access-9m5wz\") pod \"maas-controller-7f954f7b5-z94c4\" (UID: \"c01cada3-db6d-4757-8a5c-96c6296d3088\") " pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:41.783782 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.783693 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c01cada3-db6d-4757-8a5c-96c6296d3088-webhook-cert\") pod \"maas-controller-7f954f7b5-z94c4\" (UID: \"c01cada3-db6d-4757-8a5c-96c6296d3088\") " pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:41.884981 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.884954 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c01cada3-db6d-4757-8a5c-96c6296d3088-webhook-cert\") pod \"maas-controller-7f954f7b5-z94c4\" (UID: \"c01cada3-db6d-4757-8a5c-96c6296d3088\") " pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:41.885084 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.885029 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m5wz\" (UniqueName: \"kubernetes.io/projected/c01cada3-db6d-4757-8a5c-96c6296d3088-kube-api-access-9m5wz\") pod \"maas-controller-7f954f7b5-z94c4\" (UID: \"c01cada3-db6d-4757-8a5c-96c6296d3088\") " pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:41.887245 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.887228 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c01cada3-db6d-4757-8a5c-96c6296d3088-webhook-cert\") pod \"maas-controller-7f954f7b5-z94c4\" (UID: \"c01cada3-db6d-4757-8a5c-96c6296d3088\") " pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:41.908060 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:41.908032 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m5wz\" (UniqueName: \"kubernetes.io/projected/c01cada3-db6d-4757-8a5c-96c6296d3088-kube-api-access-9m5wz\") pod \"maas-controller-7f954f7b5-z94c4\" (UID: \"c01cada3-db6d-4757-8a5c-96c6296d3088\") " pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:42.064583 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:42.064532 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:42.250330 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:42.250297 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-z94c4"] Aug 04 09:53:42.253460 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:53:42.253431 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc01cada3_db6d_4757_8a5c_96c6296d3088.slice/crio-f62865112608539a217e2f9980b2f93cb23833c95fe2b5aba67a2fba219b7504 WatchSource:0}: Error finding container f62865112608539a217e2f9980b2f93cb23833c95fe2b5aba67a2fba219b7504: Status 404 returned error can't find the container with id f62865112608539a217e2f9980b2f93cb23833c95fe2b5aba67a2fba219b7504 Aug 04 09:53:43.057803 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:43.057769 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f954f7b5-z94c4" event={"ID":"c01cada3-db6d-4757-8a5c-96c6296d3088","Type":"ContainerStarted","Data":"705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d"} Aug 04 09:53:43.058124 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:43.057831 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f954f7b5-z94c4" event={"ID":"c01cada3-db6d-4757-8a5c-96c6296d3088","Type":"ContainerStarted","Data":"f62865112608539a217e2f9980b2f93cb23833c95fe2b5aba67a2fba219b7504"} Aug 04 09:53:43.058124 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:43.057853 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:43.082946 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:43.082907 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f954f7b5-z94c4" podStartSLOduration=1.398255804 podStartE2EDuration="2.082893661s" totalImagesPullingTime="684.637857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:53:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:53:43.080661445 +0000 UTC m=+792.271953941" watchObservedRunningTime="2026-08-04 09:53:43.082893661 +0000 UTC m=+792.274186154" Aug 04 09:53:54.065842 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:54.065714 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:58.777989 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:58.777954 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-z94c4"] Aug 04 09:53:58.778396 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:58.778118 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f954f7b5-z94c4" podUID="c01cada3-db6d-4757-8a5c-96c6296d3088" containerName="manager" containerID="cri-o://705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d" gracePeriod=30 Aug 04 09:53:59.022551 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.022528 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:59.119444 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.119377 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c01cada3-db6d-4757-8a5c-96c6296d3088-webhook-cert\") pod \"c01cada3-db6d-4757-8a5c-96c6296d3088\" (UID: \"c01cada3-db6d-4757-8a5c-96c6296d3088\") " Aug 04 09:53:59.119444 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.119431 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9m5wz\" (UniqueName: \"kubernetes.io/projected/c01cada3-db6d-4757-8a5c-96c6296d3088-kube-api-access-9m5wz\") pod \"c01cada3-db6d-4757-8a5c-96c6296d3088\" (UID: \"c01cada3-db6d-4757-8a5c-96c6296d3088\") " Aug 04 09:53:59.121352 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.121319 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c01cada3-db6d-4757-8a5c-96c6296d3088-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c01cada3-db6d-4757-8a5c-96c6296d3088" (UID: "c01cada3-db6d-4757-8a5c-96c6296d3088"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:53:59.121455 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.121411 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c01cada3-db6d-4757-8a5c-96c6296d3088-kube-api-access-9m5wz" (OuterVolumeSpecName: "kube-api-access-9m5wz") pod "c01cada3-db6d-4757-8a5c-96c6296d3088" (UID: "c01cada3-db6d-4757-8a5c-96c6296d3088"). InnerVolumeSpecName "kube-api-access-9m5wz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:53:59.139612 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.139586 2575 generic.go:356] "Generic (PLEG): container finished" podID="c01cada3-db6d-4757-8a5c-96c6296d3088" containerID="705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d" exitCode=0 Aug 04 09:53:59.139702 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.139629 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f954f7b5-z94c4" Aug 04 09:53:59.139702 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.139671 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f954f7b5-z94c4" event={"ID":"c01cada3-db6d-4757-8a5c-96c6296d3088","Type":"ContainerDied","Data":"705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d"} Aug 04 09:53:59.139790 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.139720 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f954f7b5-z94c4" event={"ID":"c01cada3-db6d-4757-8a5c-96c6296d3088","Type":"ContainerDied","Data":"f62865112608539a217e2f9980b2f93cb23833c95fe2b5aba67a2fba219b7504"} Aug 04 09:53:59.139790 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.139761 2575 scope.go:117] "RemoveContainer" containerID="705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d" Aug 04 09:53:59.149049 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.149033 2575 scope.go:117] "RemoveContainer" containerID="705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d" Aug 04 09:53:59.149286 ip-10-0-143-10 kubenswrapper[2575]: E0804 09:53:59.149269 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d\": container with ID starting with 705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d not found: ID does not exist" containerID="705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d" Aug 04 09:53:59.149343 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.149292 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d"} err="failed to get container status \"705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d\": rpc error: code = NotFound desc = could not find container \"705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d\": container with ID starting with 705d7dcf8836cd551b18b4a51743f548354ad74e97d46b5e98ca1898024ad61d not found: ID does not exist" Aug 04 09:53:59.167528 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.167500 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-z94c4"] Aug 04 09:53:59.171111 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.171090 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-z94c4"] Aug 04 09:53:59.220933 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.220910 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c01cada3-db6d-4757-8a5c-96c6296d3088-webhook-cert\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:53:59.220933 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.220933 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9m5wz\" (UniqueName: \"kubernetes.io/projected/c01cada3-db6d-4757-8a5c-96c6296d3088-kube-api-access-9m5wz\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 09:53:59.338645 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:53:59.338613 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c01cada3-db6d-4757-8a5c-96c6296d3088" path="/var/lib/kubelet/pods/c01cada3-db6d-4757-8a5c-96c6296d3088/volumes" Aug 04 09:54:00.123471 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.123434 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f954f7b5-x4wkb"] Aug 04 09:54:00.123852 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.123833 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c01cada3-db6d-4757-8a5c-96c6296d3088" containerName="manager" Aug 04 09:54:00.123852 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.123846 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c01cada3-db6d-4757-8a5c-96c6296d3088" containerName="manager" Aug 04 09:54:00.123931 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.123920 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c01cada3-db6d-4757-8a5c-96c6296d3088" containerName="manager" Aug 04 09:54:00.128331 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.128314 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f954f7b5-x4wkb" Aug 04 09:54:00.133965 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.133946 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jg7n5\"" Aug 04 09:54:00.136316 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.136300 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Aug 04 09:54:00.136821 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.136805 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Aug 04 09:54:00.155740 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.155703 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-x4wkb"] Aug 04 09:54:00.230164 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.230139 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac5104d9-75b8-429d-9f23-4448a7b20b86-webhook-cert\") pod \"maas-controller-7f954f7b5-x4wkb\" (UID: \"ac5104d9-75b8-429d-9f23-4448a7b20b86\") " pod="opendatahub/maas-controller-7f954f7b5-x4wkb" Aug 04 09:54:00.230280 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.230175 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4bbt\" (UniqueName: \"kubernetes.io/projected/ac5104d9-75b8-429d-9f23-4448a7b20b86-kube-api-access-g4bbt\") pod \"maas-controller-7f954f7b5-x4wkb\" (UID: \"ac5104d9-75b8-429d-9f23-4448a7b20b86\") " pod="opendatahub/maas-controller-7f954f7b5-x4wkb" Aug 04 09:54:00.331026 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.330985 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4bbt\" (UniqueName: \"kubernetes.io/projected/ac5104d9-75b8-429d-9f23-4448a7b20b86-kube-api-access-g4bbt\") pod \"maas-controller-7f954f7b5-x4wkb\" (UID: \"ac5104d9-75b8-429d-9f23-4448a7b20b86\") " pod="opendatahub/maas-controller-7f954f7b5-x4wkb" Aug 04 09:54:00.331219 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.331091 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac5104d9-75b8-429d-9f23-4448a7b20b86-webhook-cert\") pod \"maas-controller-7f954f7b5-x4wkb\" (UID: \"ac5104d9-75b8-429d-9f23-4448a7b20b86\") " pod="opendatahub/maas-controller-7f954f7b5-x4wkb" Aug 04 09:54:00.333394 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.333371 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac5104d9-75b8-429d-9f23-4448a7b20b86-webhook-cert\") pod \"maas-controller-7f954f7b5-x4wkb\" (UID: \"ac5104d9-75b8-429d-9f23-4448a7b20b86\") " pod="opendatahub/maas-controller-7f954f7b5-x4wkb" Aug 04 09:54:00.344569 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.344535 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4bbt\" (UniqueName: \"kubernetes.io/projected/ac5104d9-75b8-429d-9f23-4448a7b20b86-kube-api-access-g4bbt\") pod \"maas-controller-7f954f7b5-x4wkb\" (UID: \"ac5104d9-75b8-429d-9f23-4448a7b20b86\") " pod="opendatahub/maas-controller-7f954f7b5-x4wkb" Aug 04 09:54:00.438785 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.438693 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f954f7b5-x4wkb" Aug 04 09:54:00.583985 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:00.583960 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f954f7b5-x4wkb"] Aug 04 09:54:00.586522 ip-10-0-143-10 kubenswrapper[2575]: W0804 09:54:00.586496 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5104d9_75b8_429d_9f23_4448a7b20b86.slice/crio-7f2da7c7a103863f68c40a33d1475da26ed0aa4dc6ad7a112d20fe04ffc7b5b4 WatchSource:0}: Error finding container 7f2da7c7a103863f68c40a33d1475da26ed0aa4dc6ad7a112d20fe04ffc7b5b4: Status 404 returned error can't find the container with id 7f2da7c7a103863f68c40a33d1475da26ed0aa4dc6ad7a112d20fe04ffc7b5b4 Aug 04 09:54:01.152261 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:01.152226 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f954f7b5-x4wkb" event={"ID":"ac5104d9-75b8-429d-9f23-4448a7b20b86","Type":"ContainerStarted","Data":"7f2da7c7a103863f68c40a33d1475da26ed0aa4dc6ad7a112d20fe04ffc7b5b4"} Aug 04 09:54:02.158874 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:02.158831 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f954f7b5-x4wkb" event={"ID":"ac5104d9-75b8-429d-9f23-4448a7b20b86","Type":"ContainerStarted","Data":"85eecf19356750a1cb03aed7e12b2f86c0f8ddde6382010977f0a2efbc350951"} Aug 04 09:54:02.159332 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:02.158900 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f954f7b5-x4wkb" Aug 04 09:54:02.196878 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:02.196819 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f954f7b5-x4wkb" podStartSLOduration=1.434198012 podStartE2EDuration="2.196798277s" totalImagesPullingTime="762.600265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:54:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:54:02.196476229 +0000 UTC m=+811.387768724" watchObservedRunningTime="2026-08-04 09:54:02.196798277 +0000 UTC m=+811.388090773" Aug 04 09:54:13.166267 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:54:13.166234 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f954f7b5-x4wkb" Aug 04 09:55:31.235832 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:55:31.235759 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 09:55:31.243310 ip-10-0-143-10 kubenswrapper[2575]: I0804 09:55:31.243284 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:00:31.268077 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:00:31.268050 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:00:31.273238 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:00:31.273217 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:04:53.049043 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.048963 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np"] Aug 04 10:04:53.052874 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.052851 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.055840 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.055822 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-50928061-kserve-self-signed-certs\"" Aug 04 10:04:53.080834 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.080810 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np"] Aug 04 10:04:53.209194 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.209171 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b14e41f-812a-4759-af27-aa16a4cd7650-tls-certs\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.209302 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.209216 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-model-cache\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.209302 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.209235 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-home\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.209302 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.209286 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-dshm\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.209401 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.209303 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67h52\" (UniqueName: \"kubernetes.io/projected/0b14e41f-812a-4759-af27-aa16a4cd7650-kube-api-access-67h52\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.209401 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.209321 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-tmp-dir\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.310522 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.310460 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b14e41f-812a-4759-af27-aa16a4cd7650-tls-certs\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.310522 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.310503 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-model-cache\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.310680 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.310524 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-home\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.310756 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.310700 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-dshm\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.310816 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.310786 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67h52\" (UniqueName: \"kubernetes.io/projected/0b14e41f-812a-4759-af27-aa16a4cd7650-kube-api-access-67h52\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.310955 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.310935 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-model-cache\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.311127 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.310820 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-tmp-dir\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.311207 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.311165 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-tmp-dir\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.311274 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.311217 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-home\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.312867 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.312848 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-dshm\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.313033 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.313018 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b14e41f-812a-4759-af27-aa16a4cd7650-tls-certs\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.319547 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.319528 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67h52\" (UniqueName: \"kubernetes.io/projected/0b14e41f-812a-4759-af27-aa16a4cd7650-kube-api-access-67h52\") pod \"e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.361859 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.361837 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:04:53.502034 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:53.502011 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np"] Aug 04 10:04:53.506545 ip-10-0-143-10 kubenswrapper[2575]: W0804 10:04:53.506518 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b14e41f_812a_4759_af27_aa16a4cd7650.slice/crio-d41dc95185a54791f89d0b4e59cf0a5b16a0506af665c2ac3d0a9c957a153acf WatchSource:0}: Error finding container d41dc95185a54791f89d0b4e59cf0a5b16a0506af665c2ac3d0a9c957a153acf: Status 404 returned error can't find the container with id d41dc95185a54791f89d0b4e59cf0a5b16a0506af665c2ac3d0a9c957a153acf Aug 04 10:04:54.206999 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:54.206962 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" event={"ID":"0b14e41f-812a-4759-af27-aa16a4cd7650","Type":"ContainerStarted","Data":"df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839"} Aug 04 10:04:54.207426 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:54.207007 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" event={"ID":"0b14e41f-812a-4759-af27-aa16a4cd7650","Type":"ContainerStarted","Data":"d41dc95185a54791f89d0b4e59cf0a5b16a0506af665c2ac3d0a9c957a153acf"} Aug 04 10:04:54.233183 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:04:54.233140 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" podStartSLOduration=1.233123351 podStartE2EDuration="1.233123351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:04:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:04:54.232240621 +0000 UTC m=+1463.423533122" watchObservedRunningTime="2026-08-04 10:04:54.233123351 +0000 UTC m=+1463.424415872" Aug 04 10:05:03.362519 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:03.362443 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:05:03.362519 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:03.362485 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:05:03.374284 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:03.374262 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:05:04.263577 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:04.263550 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:05:06.932199 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:06.932163 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh"] Aug 04 10:05:06.935359 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:06.935341 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:06.938274 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:06.938258 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-50928061-kserve-self-signed-certs\"" Aug 04 10:05:06.955305 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:06.955285 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh"] Aug 04 10:05:07.019029 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.019006 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tmp-dir\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.019141 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.019036 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tls-certs\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.019141 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.019119 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-home\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.019221 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.019173 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-model-cache\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.019221 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.019198 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blndd\" (UniqueName: \"kubernetes.io/projected/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-kube-api-access-blndd\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.019282 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.019262 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-dshm\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.119741 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.119704 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-dshm\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.119839 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.119763 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tmp-dir\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.119839 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.119788 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tls-certs\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.119839 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.119827 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-home\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.120014 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.119853 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-model-cache\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.120014 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.119881 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blndd\" (UniqueName: \"kubernetes.io/projected/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-kube-api-access-blndd\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.120207 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.120182 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tmp-dir\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.120324 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.120238 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-home\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.120324 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.120284 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-model-cache\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.121868 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.121848 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-dshm\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.122196 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.122181 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tls-certs\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.128649 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.128623 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blndd\" (UniqueName: \"kubernetes.io/projected/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-kube-api-access-blndd\") pod \"e2e-same-id-b-50928061-kserve-66cffcd659-nfskh\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.245106 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.245086 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:07.375961 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:07.375935 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh"] Aug 04 10:05:07.377758 ip-10-0-143-10 kubenswrapper[2575]: W0804 10:05:07.377721 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd335ab3f_38bc_4fbb_a647_059e5f3b7c4e.slice/crio-419f18911b11632986d169020aab52955ce73338312ae595f13f829c68e2a53a WatchSource:0}: Error finding container 419f18911b11632986d169020aab52955ce73338312ae595f13f829c68e2a53a: Status 404 returned error can't find the container with id 419f18911b11632986d169020aab52955ce73338312ae595f13f829c68e2a53a Aug 04 10:05:08.272858 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:08.272823 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" event={"ID":"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e","Type":"ContainerStarted","Data":"07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20"} Aug 04 10:05:08.273199 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:08.272865 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" event={"ID":"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e","Type":"ContainerStarted","Data":"419f18911b11632986d169020aab52955ce73338312ae595f13f829c68e2a53a"} Aug 04 10:05:08.299885 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:08.299846 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" podStartSLOduration=2.299832096 podStartE2EDuration="2.299832096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:05:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:05:08.299111413 +0000 UTC m=+1477.490403908" watchObservedRunningTime="2026-08-04 10:05:08.299832096 +0000 UTC m=+1477.491124593" Aug 04 10:05:17.245638 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:17.245606 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:17.248074 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:17.245875 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:17.257842 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:17.257824 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:17.330123 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:17.330102 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:05:31.300057 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:31.300031 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:05:31.302838 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:31.302815 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:05:54.453896 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:54.453864 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np"] Aug 04 10:05:56.257021 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:05:56.256992 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh"] Aug 04 10:06:09.472112 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.472075 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" podUID="0b14e41f-812a-4759-af27-aa16a4cd7650" containerName="main" containerID="cri-o://df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839" gracePeriod=45 Aug 04 10:06:09.716819 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.716798 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:06:09.802721 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.802664 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b14e41f-812a-4759-af27-aa16a4cd7650-tls-certs\") pod \"0b14e41f-812a-4759-af27-aa16a4cd7650\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " Aug 04 10:06:09.802721 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.802716 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-model-cache\") pod \"0b14e41f-812a-4759-af27-aa16a4cd7650\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " Aug 04 10:06:09.802935 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.802763 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-dshm\") pod \"0b14e41f-812a-4759-af27-aa16a4cd7650\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " Aug 04 10:06:09.802935 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.802779 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-tmp-dir\") pod \"0b14e41f-812a-4759-af27-aa16a4cd7650\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " Aug 04 10:06:09.802935 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.802834 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67h52\" (UniqueName: \"kubernetes.io/projected/0b14e41f-812a-4759-af27-aa16a4cd7650-kube-api-access-67h52\") pod \"0b14e41f-812a-4759-af27-aa16a4cd7650\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " Aug 04 10:06:09.802935 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.802867 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-home\") pod \"0b14e41f-812a-4759-af27-aa16a4cd7650\" (UID: \"0b14e41f-812a-4759-af27-aa16a4cd7650\") " Aug 04 10:06:09.803151 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.803072 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-model-cache" (OuterVolumeSpecName: "model-cache") pod "0b14e41f-812a-4759-af27-aa16a4cd7650" (UID: "0b14e41f-812a-4759-af27-aa16a4cd7650"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:09.803151 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.803066 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0b14e41f-812a-4759-af27-aa16a4cd7650" (UID: "0b14e41f-812a-4759-af27-aa16a4cd7650"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:09.803151 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.803135 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-home" (OuterVolumeSpecName: "home") pod "0b14e41f-812a-4759-af27-aa16a4cd7650" (UID: "0b14e41f-812a-4759-af27-aa16a4cd7650"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:09.804867 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.804844 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b14e41f-812a-4759-af27-aa16a4cd7650-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0b14e41f-812a-4759-af27-aa16a4cd7650" (UID: "0b14e41f-812a-4759-af27-aa16a4cd7650"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:06:09.804975 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.804872 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-dshm" (OuterVolumeSpecName: "dshm") pod "0b14e41f-812a-4759-af27-aa16a4cd7650" (UID: "0b14e41f-812a-4759-af27-aa16a4cd7650"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:09.804975 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.804905 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b14e41f-812a-4759-af27-aa16a4cd7650-kube-api-access-67h52" (OuterVolumeSpecName: "kube-api-access-67h52") pod "0b14e41f-812a-4759-af27-aa16a4cd7650" (UID: "0b14e41f-812a-4759-af27-aa16a4cd7650"). InnerVolumeSpecName "kube-api-access-67h52". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:06:09.903817 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.903793 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-dshm\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:09.903817 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.903812 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-tmp-dir\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:09.903945 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.903822 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67h52\" (UniqueName: \"kubernetes.io/projected/0b14e41f-812a-4759-af27-aa16a4cd7650-kube-api-access-67h52\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:09.903945 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.903831 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-home\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:09.903945 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.903839 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b14e41f-812a-4759-af27-aa16a4cd7650-tls-certs\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:09.903945 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:09.903847 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b14e41f-812a-4759-af27-aa16a4cd7650-model-cache\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:10.567545 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:10.567513 2575 generic.go:356] "Generic (PLEG): container finished" podID="0b14e41f-812a-4759-af27-aa16a4cd7650" containerID="df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839" exitCode=0 Aug 04 10:06:10.567989 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:10.567549 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" event={"ID":"0b14e41f-812a-4759-af27-aa16a4cd7650","Type":"ContainerDied","Data":"df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839"} Aug 04 10:06:10.567989 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:10.567575 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" event={"ID":"0b14e41f-812a-4759-af27-aa16a4cd7650","Type":"ContainerDied","Data":"d41dc95185a54791f89d0b4e59cf0a5b16a0506af665c2ac3d0a9c957a153acf"} Aug 04 10:06:10.567989 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:10.567576 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np" Aug 04 10:06:10.567989 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:10.567592 2575 scope.go:117] "RemoveContainer" containerID="df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839" Aug 04 10:06:10.575792 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:10.575775 2575 scope.go:117] "RemoveContainer" containerID="df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839" Aug 04 10:06:10.576039 ip-10-0-143-10 kubenswrapper[2575]: E0804 10:06:10.576019 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839\": container with ID starting with df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839 not found: ID does not exist" containerID="df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839" Aug 04 10:06:10.576105 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:10.576047 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839"} err="failed to get container status \"df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839\": rpc error: code = NotFound desc = could not find container \"df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839\": container with ID starting with df7592d11ccfdd0b734a49ebddb3a020f0fd1c10935df3d10fd16b180a692839 not found: ID does not exist" Aug 04 10:06:10.594206 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:10.594179 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np"] Aug 04 10:06:10.601878 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:10.601858 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-50928061-kserve-6d8d7788ff-5x4np"] Aug 04 10:06:11.275152 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.275115 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" podUID="d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" containerName="main" containerID="cri-o://07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20" gracePeriod=45 Aug 04 10:06:11.338659 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.338633 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b14e41f-812a-4759-af27-aa16a4cd7650" path="/var/lib/kubelet/pods/0b14e41f-812a-4759-af27-aa16a4cd7650/volumes" Aug 04 10:06:11.526633 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.526584 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:06:11.573085 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.573057 2575 generic.go:356] "Generic (PLEG): container finished" podID="d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" containerID="07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20" exitCode=0 Aug 04 10:06:11.573482 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.573112 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" Aug 04 10:06:11.573482 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.573135 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" event={"ID":"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e","Type":"ContainerDied","Data":"07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20"} Aug 04 10:06:11.573482 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.573180 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh" event={"ID":"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e","Type":"ContainerDied","Data":"419f18911b11632986d169020aab52955ce73338312ae595f13f829c68e2a53a"} Aug 04 10:06:11.573482 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.573204 2575 scope.go:117] "RemoveContainer" containerID="07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20" Aug 04 10:06:11.582133 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.582111 2575 scope.go:117] "RemoveContainer" containerID="07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20" Aug 04 10:06:11.582360 ip-10-0-143-10 kubenswrapper[2575]: E0804 10:06:11.582344 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20\": container with ID starting with 07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20 not found: ID does not exist" containerID="07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20" Aug 04 10:06:11.582398 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.582371 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20"} err="failed to get container status \"07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20\": rpc error: code = NotFound desc = could not find container \"07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20\": container with ID starting with 07fc830bfb0ddab64d0e07b25f30b24caffeb9b995a5034e0d8d9d2e042e0f20 not found: ID does not exist" Aug 04 10:06:11.617189 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617166 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-home\") pod \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " Aug 04 10:06:11.617346 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617209 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blndd\" (UniqueName: \"kubernetes.io/projected/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-kube-api-access-blndd\") pod \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " Aug 04 10:06:11.617346 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617234 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-dshm\") pod \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " Aug 04 10:06:11.617477 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617385 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tmp-dir\") pod \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " Aug 04 10:06:11.617477 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617394 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-home" (OuterVolumeSpecName: "home") pod "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" (UID: "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:11.617477 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617461 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tls-certs\") pod \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " Aug 04 10:06:11.617636 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617508 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-model-cache\") pod \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\" (UID: \"d335ab3f-38bc-4fbb-a647-059e5f3b7c4e\") " Aug 04 10:06:11.617692 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617632 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" (UID: "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:11.617838 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617803 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" (UID: "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:11.617967 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617924 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-model-cache\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:11.617967 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617941 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-home\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:11.617967 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.617953 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tmp-dir\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:11.619765 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.619721 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-dshm" (OuterVolumeSpecName: "dshm") pod "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" (UID: "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:06:11.619860 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.619783 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" (UID: "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:06:11.619929 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.619883 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-kube-api-access-blndd" (OuterVolumeSpecName: "kube-api-access-blndd") pod "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" (UID: "d335ab3f-38bc-4fbb-a647-059e5f3b7c4e"). InnerVolumeSpecName "kube-api-access-blndd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:06:11.719346 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.719314 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blndd\" (UniqueName: \"kubernetes.io/projected/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-kube-api-access-blndd\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:11.719346 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.719344 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-dshm\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:11.719475 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.719355 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e-tls-certs\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:06:11.897814 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.897788 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh"] Aug 04 10:06:11.906868 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:11.906849 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-50928061-kserve-66cffcd659-nfskh"] Aug 04 10:06:13.337722 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:06:13.337684 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" path="/var/lib/kubelet/pods/d335ab3f-38bc-4fbb-a647-059e5f3b7c4e/volumes" Aug 04 10:10:31.331358 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:10:31.331332 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:10:31.331800 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:10:31.331749 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:15:31.362877 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:15:31.362853 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:15:31.365315 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:15:31.362888 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:20:15.705292 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.705262 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp"] Aug 04 10:20:15.705807 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.705665 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b14e41f-812a-4759-af27-aa16a4cd7650" containerName="main" Aug 04 10:20:15.705807 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.705676 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b14e41f-812a-4759-af27-aa16a4cd7650" containerName="main" Aug 04 10:20:15.705807 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.705703 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" containerName="main" Aug 04 10:20:15.705807 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.705708 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" containerName="main" Aug 04 10:20:15.705807 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.705787 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b14e41f-812a-4759-af27-aa16a4cd7650" containerName="main" Aug 04 10:20:15.705807 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.705799 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d335ab3f-38bc-4fbb-a647-059e5f3b7c4e" containerName="main" Aug 04 10:20:15.708754 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.708722 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:15.711106 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.711086 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9bae9cd1-serving-cert\"" Aug 04 10:20:15.724044 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.724022 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp"] Aug 04 10:20:15.816233 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.816209 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27effcdb-87ed-4451-bf0e-a495885e1c58-maas-api-tls\") pod \"maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:15.816342 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.816254 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27effcdb-87ed-4451-bf0e-a495885e1c58-openshift-service-ca\") pod \"maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:15.816390 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.816339 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2977\" (UniqueName: \"kubernetes.io/projected/27effcdb-87ed-4451-bf0e-a495885e1c58-kube-api-access-c2977\") pod \"maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:15.916852 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.916832 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27effcdb-87ed-4451-bf0e-a495885e1c58-maas-api-tls\") pod \"maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:15.916951 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.916870 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27effcdb-87ed-4451-bf0e-a495885e1c58-openshift-service-ca\") pod \"maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:15.916951 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.916915 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2977\" (UniqueName: \"kubernetes.io/projected/27effcdb-87ed-4451-bf0e-a495885e1c58-kube-api-access-c2977\") pod \"maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:15.917031 ip-10-0-143-10 kubenswrapper[2575]: E0804 10:20:15.916966 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-serving-cert: secret "maas-api-e2e-ait-9bae9cd1-serving-cert" not found Aug 04 10:20:15.917068 ip-10-0-143-10 kubenswrapper[2575]: E0804 10:20:15.917039 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27effcdb-87ed-4451-bf0e-a495885e1c58-maas-api-tls podName:27effcdb-87ed-4451-bf0e-a495885e1c58 nodeName:}" failed. No retries permitted until 2026-08-04 10:20:16.417023715 +0000 UTC m=+2385.608316189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/27effcdb-87ed-4451-bf0e-a495885e1c58-maas-api-tls") pod "maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" (UID: "27effcdb-87ed-4451-bf0e-a495885e1c58") : secret "maas-api-e2e-ait-9bae9cd1-serving-cert" not found Aug 04 10:20:15.917527 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.917511 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27effcdb-87ed-4451-bf0e-a495885e1c58-openshift-service-ca\") pod \"maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:15.927460 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:15.927432 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2977\" (UniqueName: \"kubernetes.io/projected/27effcdb-87ed-4451-bf0e-a495885e1c58-kube-api-access-c2977\") pod \"maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:16.419706 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.419663 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27effcdb-87ed-4451-bf0e-a495885e1c58-maas-api-tls\") pod \"maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:16.421841 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.421821 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27effcdb-87ed-4451-bf0e-a495885e1c58-maas-api-tls\") pod \"maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:16.618152 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.618124 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r"] Aug 04 10:20:16.618298 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.618158 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:16.621354 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.621336 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:20:16.623839 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.623819 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-9bae9cd1-dockercfg-s7bww\"" Aug 04 10:20:16.624154 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.624137 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9bae9cd1\"" Aug 04 10:20:16.646776 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.646741 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r"] Aug 04 10:20:16.707386 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.707361 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr"] Aug 04 10:20:16.711187 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.711166 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:20:16.721888 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.721864 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhdx9\" (UniqueName: \"kubernetes.io/projected/45054a78-c0d0-4225-b3ac-40f730f135ed-kube-api-access-lhdx9\") pod \"payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r\" (UID: \"45054a78-c0d0-4225-b3ac-40f730f135ed\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:20:16.722010 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.721914 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45054a78-c0d0-4225-b3ac-40f730f135ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r\" (UID: \"45054a78-c0d0-4225-b3ac-40f730f135ed\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:20:16.736868 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.736842 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr"] Aug 04 10:20:16.790383 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.790318 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp"] Aug 04 10:20:16.793292 ip-10-0-143-10 kubenswrapper[2575]: W0804 10:20:16.793269 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27effcdb_87ed_4451_bf0e_a495885e1c58.slice/crio-a30dff484539b78eff268530586dfab6cef32e0645eb768ecce26eb92c34b6ea WatchSource:0}: Error finding container a30dff484539b78eff268530586dfab6cef32e0645eb768ecce26eb92c34b6ea: Status 404 returned error can't find the container with id a30dff484539b78eff268530586dfab6cef32e0645eb768ecce26eb92c34b6ea Aug 04 10:20:16.795026 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.795008 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 10:20:16.823075 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.823053 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45054a78-c0d0-4225-b3ac-40f730f135ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r\" (UID: \"45054a78-c0d0-4225-b3ac-40f730f135ed\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:20:16.823178 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.823085 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440e1a3b-7963-438a-9e06-2c17489bdf71-plugins-config-volume\") pod \"payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr\" (UID: \"440e1a3b-7963-438a-9e06-2c17489bdf71\") " pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:20:16.823178 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.823125 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mktcd\" (UniqueName: \"kubernetes.io/projected/440e1a3b-7963-438a-9e06-2c17489bdf71-kube-api-access-mktcd\") pod \"payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr\" (UID: \"440e1a3b-7963-438a-9e06-2c17489bdf71\") " pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:20:16.823178 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.823161 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhdx9\" (UniqueName: \"kubernetes.io/projected/45054a78-c0d0-4225-b3ac-40f730f135ed-kube-api-access-lhdx9\") pod \"payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r\" (UID: \"45054a78-c0d0-4225-b3ac-40f730f135ed\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:20:16.823723 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.823706 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45054a78-c0d0-4225-b3ac-40f730f135ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r\" (UID: \"45054a78-c0d0-4225-b3ac-40f730f135ed\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:20:16.835243 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.835224 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhdx9\" (UniqueName: \"kubernetes.io/projected/45054a78-c0d0-4225-b3ac-40f730f135ed-kube-api-access-lhdx9\") pod \"payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r\" (UID: \"45054a78-c0d0-4225-b3ac-40f730f135ed\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:20:16.924279 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.924208 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440e1a3b-7963-438a-9e06-2c17489bdf71-plugins-config-volume\") pod \"payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr\" (UID: \"440e1a3b-7963-438a-9e06-2c17489bdf71\") " pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:20:16.924279 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.924263 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mktcd\" (UniqueName: \"kubernetes.io/projected/440e1a3b-7963-438a-9e06-2c17489bdf71-kube-api-access-mktcd\") pod \"payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr\" (UID: \"440e1a3b-7963-438a-9e06-2c17489bdf71\") " pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:20:16.924869 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.924848 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440e1a3b-7963-438a-9e06-2c17489bdf71-plugins-config-volume\") pod \"payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr\" (UID: \"440e1a3b-7963-438a-9e06-2c17489bdf71\") " pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:20:16.937997 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.937972 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mktcd\" (UniqueName: \"kubernetes.io/projected/440e1a3b-7963-438a-9e06-2c17489bdf71-kube-api-access-mktcd\") pod \"payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr\" (UID: \"440e1a3b-7963-438a-9e06-2c17489bdf71\") " pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:20:16.948660 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:16.948638 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:20:17.023815 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.023787 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:20:17.107349 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.107224 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r"] Aug 04 10:20:17.110293 ip-10-0-143-10 kubenswrapper[2575]: W0804 10:20:17.110265 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45054a78_c0d0_4225_b3ac_40f730f135ed.slice/crio-db59fcf7a81461f6a1e1b8d5bfd6452f4e71c3805e7929708954be526efc76d6 WatchSource:0}: Error finding container db59fcf7a81461f6a1e1b8d5bfd6452f4e71c3805e7929708954be526efc76d6: Status 404 returned error can't find the container with id db59fcf7a81461f6a1e1b8d5bfd6452f4e71c3805e7929708954be526efc76d6 Aug 04 10:20:17.203396 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.203369 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr"] Aug 04 10:20:17.204807 ip-10-0-143-10 kubenswrapper[2575]: W0804 10:20:17.204778 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440e1a3b_7963_438a_9e06_2c17489bdf71.slice/crio-19c5c54c061804c31745804039111426da587cf94bc2e7eefef6392b7d63bbfa WatchSource:0}: Error finding container 19c5c54c061804c31745804039111426da587cf94bc2e7eefef6392b7d63bbfa: Status 404 returned error can't find the container with id 19c5c54c061804c31745804039111426da587cf94bc2e7eefef6392b7d63bbfa Aug 04 10:20:17.445088 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.445030 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" event={"ID":"27effcdb-87ed-4451-bf0e-a495885e1c58","Type":"ContainerStarted","Data":"2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28"} Aug 04 10:20:17.445266 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.445201 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" event={"ID":"27effcdb-87ed-4451-bf0e-a495885e1c58","Type":"ContainerStarted","Data":"a30dff484539b78eff268530586dfab6cef32e0645eb768ecce26eb92c34b6ea"} Aug 04 10:20:17.445266 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.445237 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:17.447234 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.447182 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" event={"ID":"440e1a3b-7963-438a-9e06-2c17489bdf71","Type":"ContainerStarted","Data":"4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988"} Aug 04 10:20:17.447234 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.447227 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" event={"ID":"440e1a3b-7963-438a-9e06-2c17489bdf71","Type":"ContainerStarted","Data":"19c5c54c061804c31745804039111426da587cf94bc2e7eefef6392b7d63bbfa"} Aug 04 10:20:17.447404 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.447254 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:20:17.448958 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.448936 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" event={"ID":"45054a78-c0d0-4225-b3ac-40f730f135ed","Type":"ContainerStarted","Data":"bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0"} Aug 04 10:20:17.449084 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.448981 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" event={"ID":"45054a78-c0d0-4225-b3ac-40f730f135ed","Type":"ContainerStarted","Data":"db59fcf7a81461f6a1e1b8d5bfd6452f4e71c3805e7929708954be526efc76d6"} Aug 04 10:20:17.449084 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.449014 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:20:17.472262 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.472215 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" podStartSLOduration=1.9571908310000001 podStartE2EDuration="2.472197859s" totalImagesPullingTime="515.007028ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:20:17.471047724 +0000 UTC m=+2386.662340219" watchObservedRunningTime="2026-08-04 10:20:17.472197859 +0000 UTC m=+2386.663490360" Aug 04 10:20:17.495817 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.495782 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" podStartSLOduration=1.495770717 podStartE2EDuration="1.495770717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:20:17.49368638 +0000 UTC m=+2386.684978893" watchObservedRunningTime="2026-08-04 10:20:17.495770717 +0000 UTC m=+2386.687063219" Aug 04 10:20:17.516704 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:17.516666 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" podStartSLOduration=1.5166536609999999 podStartE2EDuration="1.516653661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:20:17.514977546 +0000 UTC m=+2386.706270043" watchObservedRunningTime="2026-08-04 10:20:17.516653661 +0000 UTC m=+2386.707946155" Aug 04 10:20:23.459410 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:23.459384 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:20:28.456348 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:28.456320 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:20:28.456832 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:28.456604 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:20:31.391892 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:31.391799 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:20:31.397177 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:20:31.393812 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:23:28.772714 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:28.772682 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp"] Aug 04 10:23:28.773214 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:28.772890 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" podUID="27effcdb-87ed-4451-bf0e-a495885e1c58" containerName="maas-api" containerID="cri-o://2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28" gracePeriod=30 Aug 04 10:23:28.869821 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:28.869795 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr"] Aug 04 10:23:28.870022 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:28.869998 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" podUID="440e1a3b-7963-438a-9e06-2c17489bdf71" containerName="payload-processing" containerID="cri-o://4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988" gracePeriod=30 Aug 04 10:23:28.876264 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:28.876199 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r"] Aug 04 10:23:28.876499 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:28.876456 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" podUID="45054a78-c0d0-4225-b3ac-40f730f135ed" containerName="payload-pre-processing" containerID="cri-o://bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0" gracePeriod=30 Aug 04 10:23:28.948924 ip-10-0-143-10 kubenswrapper[2575]: E0804 10:23:28.948891 2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45054a78_c0d0_4225_b3ac_40f730f135ed.slice/crio-bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0.scope\": RecentStats: unable to find data in memory cache]" Aug 04 10:23:29.027077 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.027001 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:23:29.125436 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.125411 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:23:29.132982 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.132963 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:23:29.176799 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.176639 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2977\" (UniqueName: \"kubernetes.io/projected/27effcdb-87ed-4451-bf0e-a495885e1c58-kube-api-access-c2977\") pod \"27effcdb-87ed-4451-bf0e-a495885e1c58\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " Aug 04 10:23:29.176799 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.176677 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27effcdb-87ed-4451-bf0e-a495885e1c58-openshift-service-ca\") pod \"27effcdb-87ed-4451-bf0e-a495885e1c58\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " Aug 04 10:23:29.176799 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.176716 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27effcdb-87ed-4451-bf0e-a495885e1c58-maas-api-tls\") pod \"27effcdb-87ed-4451-bf0e-a495885e1c58\" (UID: \"27effcdb-87ed-4451-bf0e-a495885e1c58\") " Aug 04 10:23:29.177380 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.177143 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27effcdb-87ed-4451-bf0e-a495885e1c58-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "27effcdb-87ed-4451-bf0e-a495885e1c58" (UID: "27effcdb-87ed-4451-bf0e-a495885e1c58"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:23:29.179550 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.179524 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27effcdb-87ed-4451-bf0e-a495885e1c58-kube-api-access-c2977" (OuterVolumeSpecName: "kube-api-access-c2977") pod "27effcdb-87ed-4451-bf0e-a495885e1c58" (UID: "27effcdb-87ed-4451-bf0e-a495885e1c58"). InnerVolumeSpecName "kube-api-access-c2977". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:23:29.180147 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.180115 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27effcdb-87ed-4451-bf0e-a495885e1c58-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "27effcdb-87ed-4451-bf0e-a495885e1c58" (UID: "27effcdb-87ed-4451-bf0e-a495885e1c58"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:23:29.277712 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.277641 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440e1a3b-7963-438a-9e06-2c17489bdf71-plugins-config-volume\") pod \"440e1a3b-7963-438a-9e06-2c17489bdf71\" (UID: \"440e1a3b-7963-438a-9e06-2c17489bdf71\") " Aug 04 10:23:29.277712 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.277674 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhdx9\" (UniqueName: \"kubernetes.io/projected/45054a78-c0d0-4225-b3ac-40f730f135ed-kube-api-access-lhdx9\") pod \"45054a78-c0d0-4225-b3ac-40f730f135ed\" (UID: \"45054a78-c0d0-4225-b3ac-40f730f135ed\") " Aug 04 10:23:29.277712 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.277706 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mktcd\" (UniqueName: \"kubernetes.io/projected/440e1a3b-7963-438a-9e06-2c17489bdf71-kube-api-access-mktcd\") pod \"440e1a3b-7963-438a-9e06-2c17489bdf71\" (UID: \"440e1a3b-7963-438a-9e06-2c17489bdf71\") " Aug 04 10:23:29.277986 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.277750 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45054a78-c0d0-4225-b3ac-40f730f135ed-plugins-config-volume\") pod \"45054a78-c0d0-4225-b3ac-40f730f135ed\" (UID: \"45054a78-c0d0-4225-b3ac-40f730f135ed\") " Aug 04 10:23:29.277986 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.277953 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2977\" (UniqueName: \"kubernetes.io/projected/27effcdb-87ed-4451-bf0e-a495885e1c58-kube-api-access-c2977\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:23:29.277986 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.277964 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27effcdb-87ed-4451-bf0e-a495885e1c58-openshift-service-ca\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:23:29.277986 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.277973 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27effcdb-87ed-4451-bf0e-a495885e1c58-maas-api-tls\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:23:29.278198 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.278170 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45054a78-c0d0-4225-b3ac-40f730f135ed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "45054a78-c0d0-4225-b3ac-40f730f135ed" (UID: "45054a78-c0d0-4225-b3ac-40f730f135ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:23:29.278247 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.278173 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/440e1a3b-7963-438a-9e06-2c17489bdf71-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "440e1a3b-7963-438a-9e06-2c17489bdf71" (UID: "440e1a3b-7963-438a-9e06-2c17489bdf71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:23:29.279759 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.279737 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45054a78-c0d0-4225-b3ac-40f730f135ed-kube-api-access-lhdx9" (OuterVolumeSpecName: "kube-api-access-lhdx9") pod "45054a78-c0d0-4225-b3ac-40f730f135ed" (UID: "45054a78-c0d0-4225-b3ac-40f730f135ed"). InnerVolumeSpecName "kube-api-access-lhdx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:23:29.279825 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.279783 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/440e1a3b-7963-438a-9e06-2c17489bdf71-kube-api-access-mktcd" (OuterVolumeSpecName: "kube-api-access-mktcd") pod "440e1a3b-7963-438a-9e06-2c17489bdf71" (UID: "440e1a3b-7963-438a-9e06-2c17489bdf71"). InnerVolumeSpecName "kube-api-access-mktcd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:23:29.352012 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.351989 2575 generic.go:356] "Generic (PLEG): container finished" podID="27effcdb-87ed-4451-bf0e-a495885e1c58" containerID="2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28" exitCode=0 Aug 04 10:23:29.352094 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.352024 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" event={"ID":"27effcdb-87ed-4451-bf0e-a495885e1c58","Type":"ContainerDied","Data":"2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28"} Aug 04 10:23:29.352156 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.352050 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" Aug 04 10:23:29.352156 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.352146 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp" event={"ID":"27effcdb-87ed-4451-bf0e-a495885e1c58","Type":"ContainerDied","Data":"a30dff484539b78eff268530586dfab6cef32e0645eb768ecce26eb92c34b6ea"} Aug 04 10:23:29.352262 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.352183 2575 scope.go:117] "RemoveContainer" containerID="2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28" Aug 04 10:23:29.353547 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.353527 2575 generic.go:356] "Generic (PLEG): container finished" podID="440e1a3b-7963-438a-9e06-2c17489bdf71" containerID="4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988" exitCode=0 Aug 04 10:23:29.353638 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.353574 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" Aug 04 10:23:29.353638 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.353605 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" event={"ID":"440e1a3b-7963-438a-9e06-2c17489bdf71","Type":"ContainerDied","Data":"4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988"} Aug 04 10:23:29.353780 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.353640 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr" event={"ID":"440e1a3b-7963-438a-9e06-2c17489bdf71","Type":"ContainerDied","Data":"19c5c54c061804c31745804039111426da587cf94bc2e7eefef6392b7d63bbfa"} Aug 04 10:23:29.355132 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.355110 2575 generic.go:356] "Generic (PLEG): container finished" podID="45054a78-c0d0-4225-b3ac-40f730f135ed" containerID="bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0" exitCode=0 Aug 04 10:23:29.355243 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.355148 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" Aug 04 10:23:29.355243 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.355157 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" event={"ID":"45054a78-c0d0-4225-b3ac-40f730f135ed","Type":"ContainerDied","Data":"bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0"} Aug 04 10:23:29.355243 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.355193 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r" event={"ID":"45054a78-c0d0-4225-b3ac-40f730f135ed","Type":"ContainerDied","Data":"db59fcf7a81461f6a1e1b8d5bfd6452f4e71c3805e7929708954be526efc76d6"} Aug 04 10:23:29.363597 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.363566 2575 scope.go:117] "RemoveContainer" containerID="2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28" Aug 04 10:23:29.364025 ip-10-0-143-10 kubenswrapper[2575]: E0804 10:23:29.363900 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28\": container with ID starting with 2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28 not found: ID does not exist" containerID="2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28" Aug 04 10:23:29.364025 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.363933 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28"} err="failed to get container status \"2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28\": rpc error: code = NotFound desc = could not find container \"2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28\": container with ID starting with 2bea0fe8a9a6cbd58f2c484ed08ad47019b2f6cdfea53b3f034005ff469b2c28 not found: ID does not exist" Aug 04 10:23:29.364025 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.363955 2575 scope.go:117] "RemoveContainer" containerID="4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988" Aug 04 10:23:29.371337 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.371322 2575 scope.go:117] "RemoveContainer" containerID="4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988" Aug 04 10:23:29.371555 ip-10-0-143-10 kubenswrapper[2575]: E0804 10:23:29.371536 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988\": container with ID starting with 4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988 not found: ID does not exist" containerID="4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988" Aug 04 10:23:29.371595 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.371562 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988"} err="failed to get container status \"4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988\": rpc error: code = NotFound desc = could not find container \"4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988\": container with ID starting with 4d7147053aff51db76af408eaa2ecaad8ac8fb88ab82b67385f43025e7b9c988 not found: ID does not exist" Aug 04 10:23:29.371595 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.371579 2575 scope.go:117] "RemoveContainer" containerID="bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0" Aug 04 10:23:29.377826 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.377806 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp"] Aug 04 10:23:29.378509 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.378487 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440e1a3b-7963-438a-9e06-2c17489bdf71-plugins-config-volume\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:23:29.378611 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.378512 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhdx9\" (UniqueName: \"kubernetes.io/projected/45054a78-c0d0-4225-b3ac-40f730f135ed-kube-api-access-lhdx9\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:23:29.378611 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.378534 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mktcd\" (UniqueName: \"kubernetes.io/projected/440e1a3b-7963-438a-9e06-2c17489bdf71-kube-api-access-mktcd\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:23:29.378611 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.378547 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45054a78-c0d0-4225-b3ac-40f730f135ed-plugins-config-volume\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:23:29.380035 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.380020 2575 scope.go:117] "RemoveContainer" containerID="bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0" Aug 04 10:23:29.380263 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.380245 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9bae9cd1-5db5c669f9-44rwp"] Aug 04 10:23:29.380263 ip-10-0-143-10 kubenswrapper[2575]: E0804 10:23:29.380254 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0\": container with ID starting with bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0 not found: ID does not exist" containerID="bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0" Aug 04 10:23:29.380357 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.380273 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0"} err="failed to get container status \"bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0\": rpc error: code = NotFound desc = could not find container \"bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0\": container with ID starting with bded2d6b5447f4b6fa282fdd9327183d0bada14197cf4f94dbe3879d415cfbc0 not found: ID does not exist" Aug 04 10:23:29.398238 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.398219 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r"] Aug 04 10:23:29.408151 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.408133 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9bae9cd1-648c76cbd4-9mt2r"] Aug 04 10:23:29.421190 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.421166 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr"] Aug 04 10:23:29.425033 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:29.425016 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9bae9cd1-5d5d968544-c2lrr"] Aug 04 10:23:30.307196 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.307167 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7"] Aug 04 10:23:30.307723 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.307704 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45054a78-c0d0-4225-b3ac-40f730f135ed" containerName="payload-pre-processing" Aug 04 10:23:30.307822 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.307745 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="45054a78-c0d0-4225-b3ac-40f730f135ed" containerName="payload-pre-processing" Aug 04 10:23:30.307822 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.307760 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="440e1a3b-7963-438a-9e06-2c17489bdf71" containerName="payload-processing" Aug 04 10:23:30.307822 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.307769 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="440e1a3b-7963-438a-9e06-2c17489bdf71" containerName="payload-processing" Aug 04 10:23:30.307822 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.307788 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27effcdb-87ed-4451-bf0e-a495885e1c58" containerName="maas-api" Aug 04 10:23:30.307822 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.307795 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="27effcdb-87ed-4451-bf0e-a495885e1c58" containerName="maas-api" Aug 04 10:23:30.308068 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.307892 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="45054a78-c0d0-4225-b3ac-40f730f135ed" containerName="payload-pre-processing" Aug 04 10:23:30.308068 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.307907 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="27effcdb-87ed-4451-bf0e-a495885e1c58" containerName="maas-api" Aug 04 10:23:30.308068 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.307920 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="440e1a3b-7963-438a-9e06-2c17489bdf71" containerName="payload-processing" Aug 04 10:23:30.312520 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.312502 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.318335 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.318075 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 10:23:30.318335 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.318162 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-77db2435-gw-openshift-default-dockercfg-d8s89\"" Aug 04 10:23:30.328178 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.328152 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7"] Aug 04 10:23:30.488302 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.488277 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-envoy\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.488458 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.488312 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-credential-socket\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.488458 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.488350 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-data\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.488458 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.488395 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istiod-ca-cert\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.488458 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.488420 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-token\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.488458 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.488445 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-podinfo\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.488717 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.488547 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-socket\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.488717 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.488575 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-certs\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.488717 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.488599 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd72j\" (UniqueName: \"kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-kube-api-access-rd72j\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.589805 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.589747 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-envoy\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.589805 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.589779 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-credential-socket\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.589805 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.589802 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-data\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.590024 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.589821 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istiod-ca-cert\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.590024 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.589845 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-token\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.590024 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.589870 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-podinfo\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.590024 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.589927 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-socket\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.590024 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.589948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-certs\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.590024 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.589967 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd72j\" (UniqueName: \"kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-kube-api-access-rd72j\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.590336 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.590232 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-credential-socket\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.590336 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.590286 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-data\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.590439 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.590342 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-socket\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.590439 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.590434 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-certs\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.590581 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.590527 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istiod-ca-cert\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.591818 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.591799 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-envoy\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.592334 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.592312 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-podinfo\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.601781 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.601757 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-token\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.601905 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.601890 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd72j\" (UniqueName: \"kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-kube-api-access-rd72j\") pod \"e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.627411 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.627392 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:30.778880 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:30.778829 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7"] Aug 04 10:23:30.781505 ip-10-0-143-10 kubenswrapper[2575]: W0804 10:23:30.781474 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0fcccaa_820e_4e1b_aa00_8ea0ad4df52f.slice/crio-0285ccef46151dc3929db9a2572d9e0f3d9159a82c5e992806e32e33b4a61fa2 WatchSource:0}: Error finding container 0285ccef46151dc3929db9a2572d9e0f3d9159a82c5e992806e32e33b4a61fa2: Status 404 returned error can't find the container with id 0285ccef46151dc3929db9a2572d9e0f3d9159a82c5e992806e32e33b4a61fa2 Aug 04 10:23:31.338533 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:31.338505 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27effcdb-87ed-4451-bf0e-a495885e1c58" path="/var/lib/kubelet/pods/27effcdb-87ed-4451-bf0e-a495885e1c58/volumes" Aug 04 10:23:31.338862 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:31.338847 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="440e1a3b-7963-438a-9e06-2c17489bdf71" path="/var/lib/kubelet/pods/440e1a3b-7963-438a-9e06-2c17489bdf71/volumes" Aug 04 10:23:31.339146 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:31.339134 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45054a78-c0d0-4225-b3ac-40f730f135ed" path="/var/lib/kubelet/pods/45054a78-c0d0-4225-b3ac-40f730f135ed/volumes" Aug 04 10:23:31.367615 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:31.367592 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" event={"ID":"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f","Type":"ContainerStarted","Data":"0285ccef46151dc3929db9a2572d9e0f3d9159a82c5e992806e32e33b4a61fa2"} Aug 04 10:23:33.654383 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:33.654347 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 10:23:33.654619 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:33.654421 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 10:23:33.654619 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:33.654449 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 10:23:34.383554 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:34.383519 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" event={"ID":"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f","Type":"ContainerStarted","Data":"60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e"} Aug 04 10:23:34.407079 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:34.407031 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" podStartSLOduration=1.536393597 podStartE2EDuration="4.407016641s" totalImagesPullingTime="2.870623044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:23:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:23:34.404644231 +0000 UTC m=+2583.595936725" watchObservedRunningTime="2026-08-04 10:23:34.407016641 +0000 UTC m=+2583.598309136" Aug 04 10:23:34.628268 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:34.628240 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:34.632545 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:34.632524 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:35.387578 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:35.387547 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:35.388456 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:35.388440 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:36.076483 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.076450 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v"] Aug 04 10:23:36.080751 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.080718 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:36.082929 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.082906 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-77db2435-serving-cert\"" Aug 04 10:23:36.095076 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.095057 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v"] Aug 04 10:23:36.236259 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.236230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d29bf4f6-423c-4771-bcca-43e26b871c51-maas-api-tls\") pod \"maas-api-e2e-migrate-77db2435-5c45775986-8kk6v\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:36.236405 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.236285 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29bf4f6-423c-4771-bcca-43e26b871c51-openshift-service-ca\") pod \"maas-api-e2e-migrate-77db2435-5c45775986-8kk6v\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:36.236405 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.236303 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st7dp\" (UniqueName: \"kubernetes.io/projected/d29bf4f6-423c-4771-bcca-43e26b871c51-kube-api-access-st7dp\") pod \"maas-api-e2e-migrate-77db2435-5c45775986-8kk6v\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:36.337446 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.337381 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d29bf4f6-423c-4771-bcca-43e26b871c51-maas-api-tls\") pod \"maas-api-e2e-migrate-77db2435-5c45775986-8kk6v\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:36.337572 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.337450 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29bf4f6-423c-4771-bcca-43e26b871c51-openshift-service-ca\") pod \"maas-api-e2e-migrate-77db2435-5c45775986-8kk6v\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:36.337572 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.337476 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st7dp\" (UniqueName: \"kubernetes.io/projected/d29bf4f6-423c-4771-bcca-43e26b871c51-kube-api-access-st7dp\") pod \"maas-api-e2e-migrate-77db2435-5c45775986-8kk6v\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:36.338053 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.338035 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29bf4f6-423c-4771-bcca-43e26b871c51-openshift-service-ca\") pod \"maas-api-e2e-migrate-77db2435-5c45775986-8kk6v\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:36.339707 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.339686 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d29bf4f6-423c-4771-bcca-43e26b871c51-maas-api-tls\") pod \"maas-api-e2e-migrate-77db2435-5c45775986-8kk6v\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:36.347053 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.347029 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st7dp\" (UniqueName: \"kubernetes.io/projected/d29bf4f6-423c-4771-bcca-43e26b871c51-kube-api-access-st7dp\") pod \"maas-api-e2e-migrate-77db2435-5c45775986-8kk6v\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:36.392289 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.392270 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:36.525287 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.525262 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v"] Aug 04 10:23:36.527361 ip-10-0-143-10 kubenswrapper[2575]: W0804 10:23:36.527329 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29bf4f6_423c_4771_bcca_43e26b871c51.slice/crio-e048231f4c14d7068e2a3d712f068ac2052de85d58f48b35ad07584098fcf89e WatchSource:0}: Error finding container e048231f4c14d7068e2a3d712f068ac2052de85d58f48b35ad07584098fcf89e: Status 404 returned error can't find the container with id e048231f4c14d7068e2a3d712f068ac2052de85d58f48b35ad07584098fcf89e Aug 04 10:23:36.802683 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.802654 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2"] Aug 04 10:23:36.812376 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.812348 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:23:36.814589 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.814562 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-77db2435\"" Aug 04 10:23:36.814758 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.814610 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-77db2435-dockercfg-qhhjm\"" Aug 04 10:23:36.821757 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.821719 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2"] Aug 04 10:23:36.944270 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.944234 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2\" (UID: \"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:23:36.944432 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:36.944290 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4kr8\" (UniqueName: \"kubernetes.io/projected/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-kube-api-access-r4kr8\") pod \"payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2\" (UID: \"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:23:37.045557 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.045529 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4kr8\" (UniqueName: \"kubernetes.io/projected/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-kube-api-access-r4kr8\") pod \"payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2\" (UID: \"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:23:37.045689 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.045602 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2\" (UID: \"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:23:37.046209 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.046191 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2\" (UID: \"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:23:37.061118 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.061063 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4kr8\" (UniqueName: \"kubernetes.io/projected/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-kube-api-access-r4kr8\") pod \"payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2\" (UID: \"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:23:37.123615 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.123595 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:23:37.255589 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.255565 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2"] Aug 04 10:23:37.258941 ip-10-0-143-10 kubenswrapper[2575]: W0804 10:23:37.258918 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1adb3d0_33ce_4fb4_8f75_a10bd0d053a1.slice/crio-25a907c2023b380bff2caebad5b7dd1cae98a2c782262e3674fc1f7c44927515 WatchSource:0}: Error finding container 25a907c2023b380bff2caebad5b7dd1cae98a2c782262e3674fc1f7c44927515: Status 404 returned error can't find the container with id 25a907c2023b380bff2caebad5b7dd1cae98a2c782262e3674fc1f7c44927515 Aug 04 10:23:37.399240 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.399157 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" event={"ID":"d29bf4f6-423c-4771-bcca-43e26b871c51","Type":"ContainerStarted","Data":"cf6aaede781f272e3543a5f9993f7319f2646e6a4c320d27017a162f859c1b91"} Aug 04 10:23:37.399240 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.399209 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" event={"ID":"d29bf4f6-423c-4771-bcca-43e26b871c51","Type":"ContainerStarted","Data":"e048231f4c14d7068e2a3d712f068ac2052de85d58f48b35ad07584098fcf89e"} Aug 04 10:23:37.399240 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.399235 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:37.400877 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.400851 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" event={"ID":"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1","Type":"ContainerStarted","Data":"d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa"} Aug 04 10:23:37.400999 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.400902 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" event={"ID":"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1","Type":"ContainerStarted","Data":"25a907c2023b380bff2caebad5b7dd1cae98a2c782262e3674fc1f7c44927515"} Aug 04 10:23:37.401084 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.401066 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:23:37.429354 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.429315 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" podStartSLOduration=0.763906249 podStartE2EDuration="1.429301574s" totalImagesPullingTime="665.395325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:23:37.427675769 +0000 UTC m=+2586.618968263" watchObservedRunningTime="2026-08-04 10:23:37.429301574 +0000 UTC m=+2586.620594068" Aug 04 10:23:37.461284 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:37.461220 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" podStartSLOduration=1.461205423 podStartE2EDuration="1.461205423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:23:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:23:37.459702706 +0000 UTC m=+2586.650995202" watchObservedRunningTime="2026-08-04 10:23:37.461205423 +0000 UTC m=+2586.652497918" Aug 04 10:23:43.411746 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:43.411704 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:23:48.408880 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:48.408850 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:23:49.459870 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:49.459837 2575 generic.go:356] "Generic (PLEG): container finished" podID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerID="60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e" exitCode=0 Aug 04 10:23:49.460236 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:49.459911 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" event={"ID":"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f","Type":"ContainerDied","Data":"60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e"} Aug 04 10:23:49.460344 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:49.460237 2575 scope.go:117] "RemoveContainer" containerID="60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e" Aug 04 10:23:49.462138 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:49.462109 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 10:23:49.462235 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:49.462194 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 10:23:49.462301 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:49.462242 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 10:23:50.387832 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:50.387804 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:50.466837 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:50.466804 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" event={"ID":"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f","Type":"ContainerStarted","Data":"3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46"} Aug 04 10:23:50.628200 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:50.628167 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:50.629309 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:50.629277 2575 patch_prober.go:28] interesting pod/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Aug 04 10:23:50.629440 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:50.629328 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Aug 04 10:23:51.628500 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:51.628467 2575 patch_prober.go:28] interesting pod/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Aug 04 10:23:51.628881 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:51.628521 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Aug 04 10:23:52.628550 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:52.628514 2575 patch_prober.go:28] interesting pod/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Aug 04 10:23:52.628948 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:52.628573 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Aug 04 10:23:53.627899 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:53.627822 2575 patch_prober.go:28] interesting pod/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Aug 04 10:23:53.627899 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:53.627878 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Aug 04 10:23:54.627838 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:54.627802 2575 patch_prober.go:28] interesting pod/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Aug 04 10:23:54.628205 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:54.627853 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Aug 04 10:23:55.631357 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:55.631330 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:55.631739 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:55.631675 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:23:55.632406 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:23:55.632387 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:25:31.423345 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:25:31.423279 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:25:31.426618 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:25:31.426594 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:27:48.272320 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.272284 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v"] Aug 04 10:27:48.272844 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.272490 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" podUID="d29bf4f6-423c-4771-bcca-43e26b871c51" containerName="maas-api" containerID="cri-o://cf6aaede781f272e3543a5f9993f7319f2646e6a4c320d27017a162f859c1b91" gracePeriod=30 Aug 04 10:27:48.406920 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.406884 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" podUID="d29bf4f6-423c-4771-bcca-43e26b871c51" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.50:8443/health\": dial tcp 10.133.0.50:8443: connect: connection refused" Aug 04 10:27:48.464698 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.464667 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2"] Aug 04 10:27:48.464937 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.464915 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" podUID="f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1" containerName="payload-processing" containerID="cri-o://d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa" gracePeriod=30 Aug 04 10:27:48.566667 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.566640 2575 generic.go:356] "Generic (PLEG): container finished" podID="d29bf4f6-423c-4771-bcca-43e26b871c51" containerID="cf6aaede781f272e3543a5f9993f7319f2646e6a4c320d27017a162f859c1b91" exitCode=0 Aug 04 10:27:48.566833 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.566715 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" event={"ID":"d29bf4f6-423c-4771-bcca-43e26b871c51","Type":"ContainerDied","Data":"cf6aaede781f272e3543a5f9993f7319f2646e6a4c320d27017a162f859c1b91"} Aug 04 10:27:48.589678 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.589659 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:27:48.696884 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.696860 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:27:48.745093 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.745060 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29bf4f6-423c-4771-bcca-43e26b871c51-openshift-service-ca\") pod \"d29bf4f6-423c-4771-bcca-43e26b871c51\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " Aug 04 10:27:48.745281 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.745124 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st7dp\" (UniqueName: \"kubernetes.io/projected/d29bf4f6-423c-4771-bcca-43e26b871c51-kube-api-access-st7dp\") pod \"d29bf4f6-423c-4771-bcca-43e26b871c51\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " Aug 04 10:27:48.745281 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.745168 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d29bf4f6-423c-4771-bcca-43e26b871c51-maas-api-tls\") pod \"d29bf4f6-423c-4771-bcca-43e26b871c51\" (UID: \"d29bf4f6-423c-4771-bcca-43e26b871c51\") " Aug 04 10:27:48.745496 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.745460 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d29bf4f6-423c-4771-bcca-43e26b871c51-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d29bf4f6-423c-4771-bcca-43e26b871c51" (UID: "d29bf4f6-423c-4771-bcca-43e26b871c51"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:27:48.745619 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.745596 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29bf4f6-423c-4771-bcca-43e26b871c51-openshift-service-ca\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:48.747788 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.747758 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d29bf4f6-423c-4771-bcca-43e26b871c51-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d29bf4f6-423c-4771-bcca-43e26b871c51" (UID: "d29bf4f6-423c-4771-bcca-43e26b871c51"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:27:48.747788 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.747772 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d29bf4f6-423c-4771-bcca-43e26b871c51-kube-api-access-st7dp" (OuterVolumeSpecName: "kube-api-access-st7dp") pod "d29bf4f6-423c-4771-bcca-43e26b871c51" (UID: "d29bf4f6-423c-4771-bcca-43e26b871c51"). InnerVolumeSpecName "kube-api-access-st7dp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:27:48.846980 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.846896 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-plugins-config-volume\") pod \"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1\" (UID: \"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1\") " Aug 04 10:27:48.846980 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.846929 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4kr8\" (UniqueName: \"kubernetes.io/projected/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-kube-api-access-r4kr8\") pod \"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1\" (UID: \"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1\") " Aug 04 10:27:48.847275 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.847257 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d29bf4f6-423c-4771-bcca-43e26b871c51-maas-api-tls\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:48.847322 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.847282 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st7dp\" (UniqueName: \"kubernetes.io/projected/d29bf4f6-423c-4771-bcca-43e26b871c51-kube-api-access-st7dp\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:48.847322 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.847311 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1" (UID: "f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:27:48.848945 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.848924 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-kube-api-access-r4kr8" (OuterVolumeSpecName: "kube-api-access-r4kr8") pod "f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1" (UID: "f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1"). InnerVolumeSpecName "kube-api-access-r4kr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:27:48.948380 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.948357 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-plugins-config-volume\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:48.948380 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:48.948376 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4kr8\" (UniqueName: \"kubernetes.io/projected/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1-kube-api-access-r4kr8\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:49.069345 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.069313 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7"] Aug 04 10:27:49.069543 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.069520 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" containerID="cri-o://3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46" gracePeriod=30 Aug 04 10:27:49.572595 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.572534 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" Aug 04 10:27:49.572972 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.572529 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v" event={"ID":"d29bf4f6-423c-4771-bcca-43e26b871c51","Type":"ContainerDied","Data":"e048231f4c14d7068e2a3d712f068ac2052de85d58f48b35ad07584098fcf89e"} Aug 04 10:27:49.572972 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.572659 2575 scope.go:117] "RemoveContainer" containerID="cf6aaede781f272e3543a5f9993f7319f2646e6a4c320d27017a162f859c1b91" Aug 04 10:27:49.573908 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.573889 2575 generic.go:356] "Generic (PLEG): container finished" podID="f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1" containerID="d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa" exitCode=0 Aug 04 10:27:49.573962 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.573922 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" event={"ID":"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1","Type":"ContainerDied","Data":"d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa"} Aug 04 10:27:49.573962 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.573939 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" Aug 04 10:27:49.573962 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.573950 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2" event={"ID":"f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1","Type":"ContainerDied","Data":"25a907c2023b380bff2caebad5b7dd1cae98a2c782262e3674fc1f7c44927515"} Aug 04 10:27:49.581281 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.581265 2575 scope.go:117] "RemoveContainer" containerID="d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa" Aug 04 10:27:49.588330 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.588315 2575 scope.go:117] "RemoveContainer" containerID="d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa" Aug 04 10:27:49.588576 ip-10-0-143-10 kubenswrapper[2575]: E0804 10:27:49.588549 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa\": container with ID starting with d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa not found: ID does not exist" containerID="d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa" Aug 04 10:27:49.588626 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.588573 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa"} err="failed to get container status \"d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa\": rpc error: code = NotFound desc = could not find container \"d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa\": container with ID starting with d4b99d6b744ba6fb9a451ba5b04b0711a07b4c09138febae0f6a4de408e222fa not found: ID does not exist" Aug 04 10:27:49.592316 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.592296 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v"] Aug 04 10:27:49.598655 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.598636 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-77db2435-5c45775986-8kk6v"] Aug 04 10:27:49.617297 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.617275 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2"] Aug 04 10:27:49.625092 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.625073 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-77db2435-7b4f7944f7-25pp2"] Aug 04 10:27:49.841552 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:49.841501 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-584647fd45-87rrn_e6c93048-1081-4ce6-a996-92398a7952af/authorino/0.log" Aug 04 10:27:51.338752 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:51.338694 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d29bf4f6-423c-4771-bcca-43e26b871c51" path="/var/lib/kubelet/pods/d29bf4f6-423c-4771-bcca-43e26b871c51/volumes" Aug 04 10:27:51.339175 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:51.339035 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1" path="/var/lib/kubelet/pods/f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1/volumes" Aug 04 10:27:54.316054 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.316029 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:27:54.388174 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388119 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-data\") pod \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " Aug 04 10:27:54.388174 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388152 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-socket\") pod \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " Aug 04 10:27:54.388174 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388169 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-credential-socket\") pod \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " Aug 04 10:27:54.388395 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388185 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-certs\") pod \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " Aug 04 10:27:54.388395 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388207 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istiod-ca-cert\") pod \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " Aug 04 10:27:54.388395 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388234 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rd72j\" (UniqueName: \"kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-kube-api-access-rd72j\") pod \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " Aug 04 10:27:54.388395 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388260 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-token\") pod \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " Aug 04 10:27:54.388395 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388318 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-podinfo\") pod \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " Aug 04 10:27:54.388395 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388340 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-envoy\") pod \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\" (UID: \"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f\") " Aug 04 10:27:54.388395 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388382 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" (UID: "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:27:54.388773 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388462 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" (UID: "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:27:54.388773 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388477 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" (UID: "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:27:54.388773 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388491 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" (UID: "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:27:54.388773 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388711 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-data\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:54.388773 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388749 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-socket\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:54.388773 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388766 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-credential-socket\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:54.389085 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388779 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-workload-certs\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:54.389085 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.388811 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" (UID: "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:27:54.390545 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.390522 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-kube-api-access-rd72j" (OuterVolumeSpecName: "kube-api-access-rd72j") pod "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" (UID: "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f"). InnerVolumeSpecName "kube-api-access-rd72j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:27:54.390645 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.390549 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" (UID: "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 10:27:54.390645 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.390556 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" (UID: "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:27:54.390645 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.390585 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" (UID: "f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:27:54.489587 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.489562 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd72j\" (UniqueName: \"kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-kube-api-access-rd72j\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:54.489587 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.489585 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-token\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:54.489718 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.489600 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-podinfo\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:54.489718 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.489612 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istio-envoy\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:54.489718 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.489623 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f-istiod-ca-cert\") on node \"ip-10-0-143-10.ec2.internal\" DevicePath \"\"" Aug 04 10:27:54.602328 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.602296 2575 generic.go:356] "Generic (PLEG): container finished" podID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerID="3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46" exitCode=0 Aug 04 10:27:54.602415 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.602351 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" Aug 04 10:27:54.602415 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.602375 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" event={"ID":"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f","Type":"ContainerDied","Data":"3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46"} Aug 04 10:27:54.602484 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.602439 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7" event={"ID":"f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f","Type":"ContainerDied","Data":"0285ccef46151dc3929db9a2572d9e0f3d9159a82c5e992806e32e33b4a61fa2"} Aug 04 10:27:54.602484 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.602462 2575 scope.go:117] "RemoveContainer" containerID="3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46" Aug 04 10:27:54.611528 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.611513 2575 scope.go:117] "RemoveContainer" containerID="60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e" Aug 04 10:27:54.619384 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.619369 2575 scope.go:117] "RemoveContainer" containerID="3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46" Aug 04 10:27:54.619608 ip-10-0-143-10 kubenswrapper[2575]: E0804 10:27:54.619592 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46\": container with ID starting with 3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46 not found: ID does not exist" containerID="3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46" Aug 04 10:27:54.619657 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.619615 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46"} err="failed to get container status \"3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46\": rpc error: code = NotFound desc = could not find container \"3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46\": container with ID starting with 3be94969ccbc81528fe45242aa603c1bc0050f51a8a86976127afe5e8b2eaa46 not found: ID does not exist" Aug 04 10:27:54.619657 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.619629 2575 scope.go:117] "RemoveContainer" containerID="60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e" Aug 04 10:27:54.619881 ip-10-0-143-10 kubenswrapper[2575]: E0804 10:27:54.619865 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e\": container with ID starting with 60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e not found: ID does not exist" containerID="60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e" Aug 04 10:27:54.619933 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.619885 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e"} err="failed to get container status \"60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e\": rpc error: code = NotFound desc = could not find container \"60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e\": container with ID starting with 60bad16631da8824b417d3c96478c8ecb1b3d53a9f38cc726cd21d8b305e532e not found: ID does not exist" Aug 04 10:27:54.632935 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.632915 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7"] Aug 04 10:27:54.651318 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:54.651266 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-77db2435-gw-openshift-default-768fcdd964-xb6l7"] Aug 04 10:27:55.338562 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:55.338534 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" path="/var/lib/kubelet/pods/f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f/volumes" Aug 04 10:27:56.271204 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:56.271175 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f954f7b5-x4wkb_ac5104d9-75b8-429d-9f23-4448a7b20b86/manager/0.log" Aug 04 10:27:56.815933 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:56.815913 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb649b865-qptgt_99873edc-d283-4a3c-8033-6b6c8cd9f9a4/copy-manifests/0.log" Aug 04 10:27:56.825190 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:56.825167 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb649b865-qptgt_99873edc-d283-4a3c-8033-6b6c8cd9f9a4/manager/0.log" Aug 04 10:27:57.342480 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:57.342460 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6bc7d6b7f8-hkgnh_e983e235-7b65-40e8-86d3-65b753ae5209/maas-api/0.log" Aug 04 10:27:57.718630 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:57.718607 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8cmvd_f66e24bd-aa1e-4d93-9273-e2b35d254411/postgres/0.log" Aug 04 10:27:58.538827 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:27:58.538796 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-584647fd45-87rrn_e6c93048-1081-4ce6-a996-92398a7952af/authorino/0.log" Aug 04 10:28:00.049544 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:00.049513 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b474cb6fc-kb2gj_2d14c47a-670f-4850-be5c-fe5f43a8a525/kube-auth-proxy/0.log" Aug 04 10:28:00.315500 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:00.315423 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4v2jh_645e6a6d-0310-4e8b-9e62-8d8a29ad6486/payload-pre-processing/0.log" Aug 04 10:28:00.921473 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:00.921438 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-spxzg_c6af1930-27a1-4883-995a-390a96400f62/main/0.log" Aug 04 10:28:01.165768 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:01.165717 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-272zf_8117de6f-e290-4954-8da1-370386428f85/main/0.log" Aug 04 10:28:07.067024 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.066989 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q6f76/must-gather-7rvmr"] Aug 04 10:28:07.067374 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067347 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1" containerName="payload-processing" Aug 04 10:28:07.067374 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067357 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1" containerName="payload-processing" Aug 04 10:28:07.067374 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067372 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d29bf4f6-423c-4771-bcca-43e26b871c51" containerName="maas-api" Aug 04 10:28:07.067470 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067379 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d29bf4f6-423c-4771-bcca-43e26b871c51" containerName="maas-api" Aug 04 10:28:07.067470 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067389 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" Aug 04 10:28:07.067470 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067395 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" Aug 04 10:28:07.067470 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067451 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d29bf4f6-423c-4771-bcca-43e26b871c51" containerName="maas-api" Aug 04 10:28:07.067470 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067460 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" Aug 04 10:28:07.067470 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067467 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" Aug 04 10:28:07.067645 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067473 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1adb3d0-33ce-4fb4-8f75-a10bd0d053a1" containerName="payload-processing" Aug 04 10:28:07.067645 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067538 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" Aug 04 10:28:07.067645 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.067544 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0fcccaa-820e-4e1b-aa00-8ea0ad4df52f" containerName="istio-proxy" Aug 04 10:28:07.072346 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.072331 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q6f76/must-gather-7rvmr" Aug 04 10:28:07.081057 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.081035 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q6f76\"/\"default-dockercfg-fm545\"" Aug 04 10:28:07.081790 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.081774 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q6f76\"/\"kube-root-ca.crt\"" Aug 04 10:28:07.081892 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.081877 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q6f76\"/\"openshift-service-ca.crt\"" Aug 04 10:28:07.090309 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.090288 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q6f76/must-gather-7rvmr"] Aug 04 10:28:07.185196 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.185175 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023-must-gather-output\") pod \"must-gather-7rvmr\" (UID: \"7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023\") " pod="openshift-must-gather-q6f76/must-gather-7rvmr" Aug 04 10:28:07.185292 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.185211 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btc8d\" (UniqueName: \"kubernetes.io/projected/7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023-kube-api-access-btc8d\") pod \"must-gather-7rvmr\" (UID: \"7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023\") " pod="openshift-must-gather-q6f76/must-gather-7rvmr" Aug 04 10:28:07.286016 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.285995 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023-must-gather-output\") pod \"must-gather-7rvmr\" (UID: \"7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023\") " pod="openshift-must-gather-q6f76/must-gather-7rvmr" Aug 04 10:28:07.286110 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.286032 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btc8d\" (UniqueName: \"kubernetes.io/projected/7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023-kube-api-access-btc8d\") pod \"must-gather-7rvmr\" (UID: \"7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023\") " pod="openshift-must-gather-q6f76/must-gather-7rvmr" Aug 04 10:28:07.286358 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.286342 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023-must-gather-output\") pod \"must-gather-7rvmr\" (UID: \"7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023\") " pod="openshift-must-gather-q6f76/must-gather-7rvmr" Aug 04 10:28:07.303704 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.303684 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btc8d\" (UniqueName: \"kubernetes.io/projected/7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023-kube-api-access-btc8d\") pod \"must-gather-7rvmr\" (UID: \"7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023\") " pod="openshift-must-gather-q6f76/must-gather-7rvmr" Aug 04 10:28:07.380758 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.380682 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q6f76/must-gather-7rvmr" Aug 04 10:28:07.528976 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.528944 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q6f76/must-gather-7rvmr"] Aug 04 10:28:07.532916 ip-10-0-143-10 kubenswrapper[2575]: W0804 10:28:07.532885 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d0c7dc0_1673_4a46_9b4e_ceb3b56e6023.slice/crio-a173f04e9a9c95c5f78ca184dd08341f6f7a1d6d1693b6b5526450e376365cbb WatchSource:0}: Error finding container a173f04e9a9c95c5f78ca184dd08341f6f7a1d6d1693b6b5526450e376365cbb: Status 404 returned error can't find the container with id a173f04e9a9c95c5f78ca184dd08341f6f7a1d6d1693b6b5526450e376365cbb Aug 04 10:28:07.534708 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.534692 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 10:28:07.660853 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:07.660786 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q6f76/must-gather-7rvmr" event={"ID":"7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023","Type":"ContainerStarted","Data":"a173f04e9a9c95c5f78ca184dd08341f6f7a1d6d1693b6b5526450e376365cbb"} Aug 04 10:28:08.669876 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:08.669783 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q6f76/must-gather-7rvmr" event={"ID":"7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023","Type":"ContainerStarted","Data":"759be171de0b31c7d669404b119918db0a4803670bc460c1685ede1713be9270"} Aug 04 10:28:08.669876 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:08.669844 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q6f76/must-gather-7rvmr" event={"ID":"7d0c7dc0-1673-4a46-9b4e-ceb3b56e6023","Type":"ContainerStarted","Data":"99b05fdceaf0795ba4bfb2b3bc33ef4b665eeadfb0787f9ecae296a2766e5dd4"} Aug 04 10:28:08.699289 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:08.699243 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-q6f76/must-gather-7rvmr" podStartSLOduration=0.928049019 podStartE2EDuration="1.699227047s" totalImagesPullingTime="771.178028ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:28:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:08.696226709 +0000 UTC m=+2857.887519203" watchObservedRunningTime="2026-08-04 10:28:08.699227047 +0000 UTC m=+2857.890519541" Aug 04 10:28:10.159079 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:10.159048 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-g6s9t_a2004042-19e9-46e9-8379-ad03ca29d486/global-pull-secret-syncer/0.log" Aug 04 10:28:10.280566 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:10.280531 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lbmxb_75063389-386c-4849-9baf-c001a3c3f9d2/konnectivity-agent/0.log" Aug 04 10:28:10.429565 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:10.429489 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-10.ec2.internal_499f0d85eada0e49ab159e031fad0580/haproxy/0.log" Aug 04 10:28:15.075156 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:15.075074 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-584647fd45-87rrn_e6c93048-1081-4ce6-a996-92398a7952af/authorino/0.log" Aug 04 10:28:18.117044 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.116959 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-tmk4h_6c4a6b2c-b806-4af1-aeeb-375b73f5519e/kube-state-metrics/0.log" Aug 04 10:28:18.172394 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.172363 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-tmk4h_6c4a6b2c-b806-4af1-aeeb-375b73f5519e/kube-rbac-proxy-main/0.log" Aug 04 10:28:18.232646 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.232609 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-tmk4h_6c4a6b2c-b806-4af1-aeeb-375b73f5519e/kube-rbac-proxy-self/0.log" Aug 04 10:28:18.341212 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.341172 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-wf7mf_97f72632-fe67-4f4c-886c-52ccb66ad0e1/monitoring-plugin/0.log" Aug 04 10:28:18.387236 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.387134 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g9r95_5a87a755-5520-43e8-a8ca-5d986051ce4a/node-exporter/0.log" Aug 04 10:28:18.421129 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.421064 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g9r95_5a87a755-5520-43e8-a8ca-5d986051ce4a/kube-rbac-proxy/0.log" Aug 04 10:28:18.465940 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.465884 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g9r95_5a87a755-5520-43e8-a8ca-5d986051ce4a/init-textfile/0.log" Aug 04 10:28:18.798869 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.798837 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-rsdmk_13d57be2-3e45-40c4-b4df-f866902a5276/kube-rbac-proxy-main/0.log" Aug 04 10:28:18.842906 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.842875 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-rsdmk_13d57be2-3e45-40c4-b4df-f866902a5276/kube-rbac-proxy-self/0.log" Aug 04 10:28:18.882281 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.882249 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-rsdmk_13d57be2-3e45-40c4-b4df-f866902a5276/openshift-state-metrics/0.log" Aug 04 10:28:18.932564 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.932536 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f414f234-fd1f-4726-b761-027122ab5cd6/prometheus/0.log" Aug 04 10:28:18.970508 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:18.970477 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f414f234-fd1f-4726-b761-027122ab5cd6/config-reloader/0.log" Aug 04 10:28:19.014280 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.014254 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f414f234-fd1f-4726-b761-027122ab5cd6/thanos-sidecar/0.log" Aug 04 10:28:19.059494 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.059411 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f414f234-fd1f-4726-b761-027122ab5cd6/kube-rbac-proxy-web/0.log" Aug 04 10:28:19.096802 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.096768 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f414f234-fd1f-4726-b761-027122ab5cd6/kube-rbac-proxy/0.log" Aug 04 10:28:19.133182 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.133153 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f414f234-fd1f-4726-b761-027122ab5cd6/kube-rbac-proxy-thanos/0.log" Aug 04 10:28:19.179046 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.179017 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f414f234-fd1f-4726-b761-027122ab5cd6/init-config-reloader/0.log" Aug 04 10:28:19.300988 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.300953 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p"] Aug 04 10:28:19.305914 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.305894 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.318918 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.318895 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p"] Aug 04 10:28:19.354534 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.354500 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-gh27g_30fe3119-97af-4955-b7b5-a6128ed3fa56/prometheus-operator-admission-webhook/0.log" Aug 04 10:28:19.423896 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.423865 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b2jn\" (UniqueName: \"kubernetes.io/projected/199d269f-a261-4f8c-8e77-5aa34aa14239-kube-api-access-2b2jn\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.424073 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.423959 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-lib-modules\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.424137 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.424060 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-proc\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.424137 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.424097 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-podres\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.424231 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.424136 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-sys\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.525122 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.525079 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b2jn\" (UniqueName: \"kubernetes.io/projected/199d269f-a261-4f8c-8e77-5aa34aa14239-kube-api-access-2b2jn\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.525308 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.525187 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-lib-modules\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.525308 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.525247 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-proc\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.525308 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.525270 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-podres\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.525493 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.525308 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-sys\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.525493 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.525366 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-lib-modules\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.525493 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.525395 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-sys\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.525493 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.525364 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-proc\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.525493 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.525455 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/199d269f-a261-4f8c-8e77-5aa34aa14239-podres\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.532106 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.532085 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6964db96b7-vfn66_12726353-01f7-4908-940f-7da5a75bf5c0/thanos-query/0.log" Aug 04 10:28:19.535297 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.535275 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b2jn\" (UniqueName: \"kubernetes.io/projected/199d269f-a261-4f8c-8e77-5aa34aa14239-kube-api-access-2b2jn\") pod \"perf-node-gather-daemonset-lfn8p\" (UID: \"199d269f-a261-4f8c-8e77-5aa34aa14239\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.570234 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.570176 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6964db96b7-vfn66_12726353-01f7-4908-940f-7da5a75bf5c0/kube-rbac-proxy-web/0.log" Aug 04 10:28:19.609876 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.609852 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6964db96b7-vfn66_12726353-01f7-4908-940f-7da5a75bf5c0/kube-rbac-proxy/0.log" Aug 04 10:28:19.615879 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.615856 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:19.648497 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.648470 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6964db96b7-vfn66_12726353-01f7-4908-940f-7da5a75bf5c0/prom-label-proxy/0.log" Aug 04 10:28:19.689318 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.689114 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6964db96b7-vfn66_12726353-01f7-4908-940f-7da5a75bf5c0/kube-rbac-proxy-rules/0.log" Aug 04 10:28:19.737546 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.736171 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6964db96b7-vfn66_12726353-01f7-4908-940f-7da5a75bf5c0/kube-rbac-proxy-metrics/0.log" Aug 04 10:28:19.782351 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:19.782325 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p"] Aug 04 10:28:20.753614 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:20.753582 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" event={"ID":"199d269f-a261-4f8c-8e77-5aa34aa14239","Type":"ContainerStarted","Data":"55698a29fc729ce349368d7f14505ea71f747b5baa9c87077282d3c6664baeab"} Aug 04 10:28:20.754097 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:20.753634 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" event={"ID":"199d269f-a261-4f8c-8e77-5aa34aa14239","Type":"ContainerStarted","Data":"ae2d7e6340881550dc46b8ce3a7aa20fce8176a8cc93a342adc1789ebf790661"} Aug 04 10:28:20.754097 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:20.753714 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:20.781129 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:20.781075 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" podStartSLOduration=1.781055893 podStartE2EDuration="1.781055893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:28:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:20.779763217 +0000 UTC m=+2869.971055712" watchObservedRunningTime="2026-08-04 10:28:20.781055893 +0000 UTC m=+2869.972348394" Aug 04 10:28:20.925131 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:20.925099 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-vq486_a00b5360-ca2b-4ed4-a9ad-1d5e1539116c/networking-console-plugin/0.log" Aug 04 10:28:21.485748 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:21.485702 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/2.log" Aug 04 10:28:21.491989 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:21.491950 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-t2tlf_0215bf3b-ca0e-454c-b48e-0ec33d6a8f23/console-operator/3.log" Aug 04 10:28:23.997887 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:23.997808 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-47flm_e6adbcf4-6061-4e4c-b52f-092d5a911104/dns/0.log" Aug 04 10:28:24.048963 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:24.048937 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-47flm_e6adbcf4-6061-4e4c-b52f-092d5a911104/kube-rbac-proxy/0.log" Aug 04 10:28:24.319151 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:24.319080 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vkqkd_8612b3d8-d52c-4b28-96b1-c9202698e69d/dns-node-resolver/0.log" Aug 04 10:28:25.199389 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:25.199358 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-slfll_b16b5696-7aaa-4c86-928c-09e2f6d8f8b4/node-ca/0.log" Aug 04 10:28:26.608994 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:26.608962 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b474cb6fc-kb2gj_2d14c47a-670f-4850-be5c-fe5f43a8a525/kube-auth-proxy/0.log" Aug 04 10:28:26.739348 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:26.739311 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4v2jh_645e6a6d-0310-4e8b-9e62-8d8a29ad6486/payload-pre-processing/0.log" Aug 04 10:28:26.771616 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:26.771591 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-lfn8p" Aug 04 10:28:27.713745 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:27.713700 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-htz5h_6c66c8c6-437d-4958-b76b-544874765c73/serve-healthcheck-canary/0.log" Aug 04 10:28:28.439924 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:28.439891 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-b5jm2_bbfc0fc9-940f-4c54-a66d-ac422afa0ceb/insights-operator/0.log" Aug 04 10:28:28.440175 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:28.440158 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-b5jm2_bbfc0fc9-940f-4c54-a66d-ac422afa0ceb/insights-operator/1.log" Aug 04 10:28:28.479359 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:28.479325 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jgcrm_d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c/kube-rbac-proxy/0.log" Aug 04 10:28:28.525454 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:28.525419 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jgcrm_d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c/exporter/0.log" Aug 04 10:28:28.571103 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:28.571075 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jgcrm_d09d40a3-a8bd-4a62-af4c-bbbf6d437b3c/extractor/0.log" Aug 04 10:28:30.171412 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:30.171383 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b474cb6fc-kb2gj_2d14c47a-670f-4850-be5c-fe5f43a8a525/kube-auth-proxy/0.log" Aug 04 10:28:30.226625 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:30.226599 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4v2jh_645e6a6d-0310-4e8b-9e62-8d8a29ad6486/payload-pre-processing/0.log" Aug 04 10:28:31.546205 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:31.546163 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b474cb6fc-kb2gj_2d14c47a-670f-4850-be5c-fe5f43a8a525/kube-auth-proxy/0.log" Aug 04 10:28:31.609172 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:31.609151 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f954f7b5-x4wkb_ac5104d9-75b8-429d-9f23-4448a7b20b86/manager/0.log" Aug 04 10:28:31.692617 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:31.692589 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4v2jh_645e6a6d-0310-4e8b-9e62-8d8a29ad6486/payload-pre-processing/0.log" Aug 04 10:28:31.894122 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:31.894040 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb649b865-qptgt_99873edc-d283-4a3c-8033-6b6c8cd9f9a4/manager/0.log" Aug 04 10:28:31.919667 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:31.919644 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb649b865-qptgt_99873edc-d283-4a3c-8033-6b6c8cd9f9a4/copy-manifests/0.log" Aug 04 10:28:33.316031 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:33.315999 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78b8bf7c54-nkwzc_2b41d453-da85-4848-b88c-8228f8869f83/manager/0.log" Aug 04 10:28:33.358252 ip-10-0-143-10 kubenswrapper[2575]: I0804 10:28:33.358223 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-xrbct_17f2b900-d774-4885-a6c9-9e94a9b207af/openshift-lws-operator/0.log"