Jul 07 13:13:24.807593 ip-10-0-143-95 systemd[1]: Starting Kubernetes Kubelet... Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.272559 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279267 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279285 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279288 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279292 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279297 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279299 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279302 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:13:25.428337 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279305 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:13:25.348540 ip-10-0-143-95 systemd[1]: Started Kubernetes Kubelet. Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279308 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279311 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279314 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279317 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279320 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279322 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279325 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279328 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279331 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279333 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279336 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279338 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279350 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279353 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279355 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279358 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279360 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279364 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279367 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:13:25.436675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279369 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279372 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279375 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279378 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279381 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279384 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279387 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279390 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279392 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279395 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279398 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279401 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279404 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279409 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279413 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279416 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279419 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279423 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279426 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279429 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:13:25.439784 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279432 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279435 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279438 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279440 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279443 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279446 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279451 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279455 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279459 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279463 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279467 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279470 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279473 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279476 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279479 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279481 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279484 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279486 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279489 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:13:25.441899 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279492 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279495 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279497 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279501 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279503 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279506 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279509 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279511 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279514 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279517 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279519 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279522 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279524 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279527 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279530 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279532 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279535 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279539 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279542 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279544 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:13:25.458941 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279547 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279549 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279552 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279554 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279557 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279559 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279562 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279564 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279567 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279569 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279572 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279574 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.279577 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280350 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280357 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280363 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280366 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280369 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280372 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280375 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:13:25.459682 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280378 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280381 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280384 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280387 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280390 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280393 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280397 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280400 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280403 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280407 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280411 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280415 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280418 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280421 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280423 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280426 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280429 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280432 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280434 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:13:25.460302 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280437 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280440 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280442 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280445 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280447 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280450 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280452 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280454 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280458 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280461 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280463 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280471 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280473 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280476 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280479 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280482 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280485 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280487 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280490 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280493 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:13:25.461090 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280496 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280499 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280501 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280504 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280507 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280510 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280513 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280516 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280518 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280520 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280523 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280525 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280528 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280531 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280533 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280536 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280540 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280543 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280547 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:13:25.461794 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280550 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280553 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280556 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280560 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280566 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280570 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280574 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280577 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280579 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280582 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280585 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280588 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280591 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280593 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280596 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280598 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280601 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280604 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280607 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:13:25.462583 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280609 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280612 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280614 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280617 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280620 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280622 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280625 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280627 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280630 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280632 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280635 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280637 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280640 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280642 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.280645 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281062 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281068 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281072 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281075 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281078 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:13:25.463158 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281080 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281083 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281086 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281089 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281091 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281094 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281096 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281099 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281102 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281104 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281118 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281122 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281125 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281127 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281130 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281133 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281135 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281138 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281141 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281143 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:13:25.463876 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281146 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281148 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281151 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281154 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281157 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281159 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281162 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281166 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281168 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281172 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281175 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281178 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281180 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281183 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281186 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281188 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281191 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281194 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281196 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:13:25.464473 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281199 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281202 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281204 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281207 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281210 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281213 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281216 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281218 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281221 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281225 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281229 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281232 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281235 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281238 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281241 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281257 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281260 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281263 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281265 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281268 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:13:25.465103 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281273 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281276 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281281 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281284 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281287 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281290 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281294 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281297 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281300 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281302 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281305 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281307 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281310 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281313 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281315 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281318 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281320 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281323 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281326 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:13:25.465677 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281329 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281331 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281334 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281337 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281339 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281342 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281345 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281347 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281350 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281353 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281356 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281359 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281362 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281366 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281369 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.281373 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281468 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281478 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281487 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281494 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281504 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 13:13:25.466212 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281509 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281516 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281522 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281525 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281529 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281532 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281536 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281540 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281543 2574 flags.go:64] FLAG: --cgroup-root="" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281546 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281548 2574 flags.go:64] FLAG: --client-ca-file="" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281552 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281556 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281561 2574 flags.go:64] FLAG: --cluster-domain="" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281563 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281567 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281570 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281573 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281578 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281581 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281584 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281588 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281591 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281594 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 13:13:25.466813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281599 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281602 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281607 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281611 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281615 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281617 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281620 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281623 2574 flags.go:64] FLAG: --enable-server="true" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281626 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281631 2574 flags.go:64] FLAG: --event-burst="100" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281634 2574 flags.go:64] FLAG: --event-qps="50" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281637 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281640 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281643 2574 flags.go:64] FLAG: --eviction-hard="" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281647 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281650 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281653 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281656 2574 flags.go:64] FLAG: --eviction-soft="" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281659 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281662 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281666 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281669 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281672 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281675 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281678 2574 flags.go:64] FLAG: --feature-gates="" Jul 07 13:13:25.467540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281682 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281685 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281688 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281691 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281695 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281698 2574 flags.go:64] FLAG: --help="false" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281701 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281705 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281708 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281712 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281716 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281719 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281722 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281725 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281728 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281731 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281734 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281737 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281740 2574 flags.go:64] FLAG: --kube-reserved="" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281743 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281745 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281748 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281751 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281754 2574 flags.go:64] FLAG: --lock-file="" Jul 07 13:13:25.468281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281757 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281760 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281763 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281768 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281772 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281774 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281777 2574 flags.go:64] FLAG: --logging-format="text" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281780 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281783 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281786 2574 flags.go:64] FLAG: --manifest-url="" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281789 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281793 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281796 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281800 2574 flags.go:64] FLAG: --max-pods="110" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281803 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281808 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281811 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281815 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281818 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281821 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281824 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281832 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281835 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281839 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 13:13:25.468940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281842 2574 flags.go:64] FLAG: --pod-cidr="" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281844 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281850 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281853 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281856 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281858 2574 flags.go:64] FLAG: --port="10250" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281862 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281865 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-067a7f86ca188ab89" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281868 2574 flags.go:64] FLAG: --qos-reserved="" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281872 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281875 2574 flags.go:64] FLAG: --register-node="true" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281877 2574 flags.go:64] FLAG: --register-with-taints="" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281881 2574 flags.go:64] FLAG: --registry-burst="10" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281885 2574 flags.go:64] FLAG: --registry-qps="5" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281888 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281890 2574 flags.go:64] FLAG: --reserved-memory="" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281894 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281897 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281900 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281903 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281906 2574 flags.go:64] FLAG: --runonce="false" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281909 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281912 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281917 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281919 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 13:13:25.469805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281922 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281927 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281930 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281933 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281936 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281939 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281942 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281945 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281948 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281951 2574 flags.go:64] FLAG: --system-cgroups="" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281954 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281960 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281963 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281966 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281974 2574 flags.go:64] FLAG: --tls-min-version="" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281977 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281980 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281983 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281986 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281989 2574 flags.go:64] FLAG: --v="2" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.281993 2574 flags.go:64] FLAG: --version="false" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.282003 2574 flags.go:64] FLAG: --vmodule="" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.282007 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.282010 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282135 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:13:25.470640 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282139 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282142 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282145 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282148 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282151 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282155 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282157 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282160 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282164 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282167 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282171 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282175 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282180 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282184 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282187 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282190 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282193 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282195 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282198 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:13:25.471469 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282201 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282204 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282206 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282209 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282211 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282214 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282217 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282219 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282222 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282224 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282228 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282230 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282233 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282235 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282238 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282241 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282258 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282261 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282265 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:13:25.472071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282267 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282270 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282275 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282277 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282280 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282283 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282286 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282288 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282291 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282294 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282296 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282299 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282301 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282304 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282307 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282309 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282312 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282314 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282317 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282319 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:13:25.473997 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282322 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282325 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282327 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282330 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282333 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282336 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282339 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282341 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282344 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282347 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282349 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282353 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282357 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282359 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282364 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282367 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282370 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282372 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282375 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282377 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:13:25.474708 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282380 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282383 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282385 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282388 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282390 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282393 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282396 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282398 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282401 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282403 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282406 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282408 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282411 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282413 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282415 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282418 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282421 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282424 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282427 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:13:25.600630 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.282429 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.283367 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.290499 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.290522 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.291441 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.291452 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291516 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291522 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291526 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291529 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291532 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291535 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291538 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291543 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:13:25.601553 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291547 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291550 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291552 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291555 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291558 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291561 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291564 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291567 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291570 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291573 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291577 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291580 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291583 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291586 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291590 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291593 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291596 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291599 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291602 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291605 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:13:25.602558 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291607 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291611 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291614 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291617 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291619 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291622 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291624 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291627 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291630 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291632 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291635 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291637 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291639 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291642 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291645 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291648 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291650 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291653 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291655 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291662 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:13:25.603138 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291665 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291668 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291670 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291673 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291675 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291678 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291683 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291685 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291688 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291692 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291695 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291698 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291701 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291704 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291707 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291710 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291712 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291715 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291717 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:13:25.604283 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291720 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291723 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291725 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291728 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291730 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291733 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291735 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291738 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291740 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291743 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291745 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291748 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291752 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291755 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291758 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291760 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291763 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291766 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291768 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291772 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:13:25.604819 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291775 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291777 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291780 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291783 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291785 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291788 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291790 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291793 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291796 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291798 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291802 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291804 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.291809 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291929 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291933 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:13:25.605493 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291936 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291940 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291944 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291947 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291950 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291953 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291956 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291958 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291961 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291965 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291968 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291970 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291973 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291976 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291979 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291981 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291983 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291987 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291990 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291993 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:13:25.605919 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291995 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.291998 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292001 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292003 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292006 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292008 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292012 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292014 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292017 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292019 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292021 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292024 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292026 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292031 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292034 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292037 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292040 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292043 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292046 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:13:25.606563 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292050 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292053 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292056 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292059 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292061 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292064 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292066 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292069 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292072 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292074 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292078 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292081 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292085 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292088 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292091 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292094 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292097 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292100 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292102 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:13:25.727359 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292105 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292108 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292111 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292113 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292116 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292118 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292121 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292123 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292126 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292128 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292131 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292133 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292136 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292138 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292141 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292145 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292148 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292150 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292153 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292155 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:13:25.728014 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292158 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292160 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292163 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292167 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292169 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292172 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292174 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292177 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292180 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292182 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292185 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292188 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292191 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292193 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292196 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292199 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292201 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292204 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:13:25.728619 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:25.292206 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.292211 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.292365 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.294462 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.296143 2574 server.go:1019] "Starting client certificate rotation" Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.296265 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.296307 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.298998 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.303275 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.311489 2574 log.go:25] "Validated CRI v1 runtime API" Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.314570 2574 log.go:25] "Validated CRI v1 image API" Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.316219 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 13:13:25.729210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.318905 2574 fs.go:135] Filesystem UUIDs: map[4fcb3c9a-0cbf-4854-98a8-bca0762a4ef8:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8b3da367-d6ca-49ee-a612-f2f5bb4b01e6:/dev/nvme0n1p3] Jul 07 13:13:25.729635 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.318929 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 13:13:25.729741 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.324431 2574 manager.go:216] Machine: {Timestamp:2026-07-07 13:13:25.322838652 +0000 UTC m=+0.377260313 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100822 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ace5b31fe721d3e832eedf1fc2fd0 SystemUUID:ec2ace5b-31fe-721d-3e83-2eedf1fc2fd0 BootID:20a555c8-ca54-449c-b37b-9261a7fc28fe Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:eb:c4:62:20:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:eb:c4:62:20:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:28:2b:fa:90:a9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 13:13:25.729741 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.324584 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 13:13:25.729741 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.324722 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 13:13:25.729741 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.328041 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 13:13:25.729741 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.328085 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-95.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 13:13:25.729741 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.328810 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 13:13:25.729741 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.328830 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 13:13:25.729741 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.328839 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.329625 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.330714 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.330987 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.333891 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.334065 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.334103 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.334124 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.334139 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.334153 2574 kubelet.go:398] "Adding apiserver pod source" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.334167 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.334714 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.334734 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.337730 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.340272 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.340305 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342197 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342214 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342220 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342226 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342235 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342241 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342265 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342271 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342278 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342283 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342293 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.342298 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.343025 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.343034 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.346665 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-95.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.347029 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.347320 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-95.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.347629 2574 server.go:1268] "Started kubelet" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.347700 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.347747 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.347838 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.349080 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.349101 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 07 13:13:25.730193 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.353854 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.354746 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.354749 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.354866 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.355242 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.355284 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.355299 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.355405 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.355416 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.356207 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.356222 2574 factory.go:55] Registering systemd factory Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.356232 2574 factory.go:223] Registration of the systemd container factory successfully Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.356455 2574 factory.go:153] Registering CRI-O factory Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.356466 2574 factory.go:223] Registration of the crio container factory successfully Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.356484 2574 factory.go:103] Registering Raw factory Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.356494 2574 manager.go:1196] Started watching for new ooms in manager Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.356952 2574 manager.go:349] Starting recovery of all containers Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.357096 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-95.ec2.internal.18c004240e842fcc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-95.ec2.internal,UID:ip-10-0-143-95.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-95.ec2.internal,},FirstTimestamp:2026-07-07 13:13:25.347594188 +0000 UTC m=+0.402015837,LastTimestamp:2026-07-07 13:13:25.347594188 +0000 UTC m=+0.402015837,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-95.ec2.internal,}" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.358185 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.358196 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-95.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.368333 2574 manager.go:354] Recovery completed Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.376966 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.379311 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.379353 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.379368 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.380095 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.380105 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.380108 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.380172 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.380946 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.380975 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.380985 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.381529 2574 policy_none.go:49] "None policy: Start" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.381546 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 13:13:25.731242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.381560 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.381577 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-95.ec2.internal.18c00424106880e5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-95.ec2.internal,UID:ip-10-0-143-95.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-95.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-95.ec2.internal,},FirstTimestamp:2026-07-07 13:13:25.379334373 +0000 UTC m=+0.433756026,LastTimestamp:2026-07-07 13:13:25.379334373 +0000 UTC m=+0.433756026,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-95.ec2.internal,}" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.382562 2574 policy_none.go:47] "Start" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.388717 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5v9s4" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.390469 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-95.ec2.internal.18c004241068e72a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-95.ec2.internal,UID:ip-10-0-143-95.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-95.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-95.ec2.internal,},FirstTimestamp:2026-07-07 13:13:25.379360554 +0000 UTC m=+0.433782204,LastTimestamp:2026-07-07 13:13:25.379360554 +0000 UTC m=+0.433782204,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-95.ec2.internal,}" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.397286 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5v9s4" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.420001 2574 manager.go:337] "Starting Device Plugin manager" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.420045 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.420059 2574 server.go:87] "Starting device plugin registration server" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.420388 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.420402 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.420593 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.420703 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.420712 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.421133 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.421185 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.423919 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.425498 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.425533 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.425555 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.425590 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.428544 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.521102 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.522545 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.522576 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.522589 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:13:25.732266 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.522620 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.526655 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-95.ec2.internal"] Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.526732 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.528613 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.528644 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.528658 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.530020 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.530134 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.530166 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.531134 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.531142 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.531160 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.531170 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.531170 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.531240 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.532375 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.532401 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.533174 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.533199 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.533215 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.540898 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.540918 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-95.ec2.internal\": node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.556889 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18e3cb94201ad665b039cc13756723f0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-95.ec2.internal\" (UID: \"18e3cb94201ad665b039cc13756723f0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.556936 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73b8060886d9cb84dfad3d715baebfe6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal\" (UID: \"73b8060886d9cb84dfad3d715baebfe6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.556953 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73b8060886d9cb84dfad3d715baebfe6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal\" (UID: \"73b8060886d9cb84dfad3d715baebfe6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.558383 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.565281 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-95.ec2.internal\" not found" node="ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733023 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.570324 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-95.ec2.internal\" not found" node="ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.657711 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73b8060886d9cb84dfad3d715baebfe6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal\" (UID: \"73b8060886d9cb84dfad3d715baebfe6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.657745 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73b8060886d9cb84dfad3d715baebfe6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal\" (UID: \"73b8060886d9cb84dfad3d715baebfe6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.657763 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18e3cb94201ad665b039cc13756723f0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-95.ec2.internal\" (UID: \"18e3cb94201ad665b039cc13756723f0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.657818 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18e3cb94201ad665b039cc13756723f0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-95.ec2.internal\" (UID: \"18e3cb94201ad665b039cc13756723f0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.657823 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73b8060886d9cb84dfad3d715baebfe6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal\" (UID: \"73b8060886d9cb84dfad3d715baebfe6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.657853 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73b8060886d9cb84dfad3d715baebfe6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal\" (UID: \"73b8060886d9cb84dfad3d715baebfe6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.733940 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.658704 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:25.860391 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.758786 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:25.860391 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.859485 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:25.991545 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.868647 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.991545 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:25.872786 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-95.ec2.internal" Jul 07 13:13:25.991545 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:25.960463 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:26.123147 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:26.060905 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:26.234955 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:26.161916 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:26.351015 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:26.262415 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:26.351015 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.296261 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 13:13:26.351015 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.296439 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 13:13:26.351015 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.313027 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:13:26.507487 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:26.363371 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:26.507487 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.401046 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 13:08:25 +0000 UTC" deadline="2028-02-22 19:30:32.633644118 +0000 UTC" Jul 07 13:13:26.507487 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.401097 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14286h17m6.232550905s" Jul 07 13:13:26.507487 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:26.463822 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:26.607227 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:26.564430 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:26.607227 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.571139 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:13:26.607227 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.572919 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:13:26.607227 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.572951 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:13:26.607227 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.572962 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:13:26.727742 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:26.665576 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-95.ec2.internal\" not found" Jul 07 13:13:26.727742 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.708475 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:13:26.883765 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:26.751395 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e3cb94201ad665b039cc13756723f0.slice/crio-b479f40556e964a00a8068333cbb0a4e2d81ff5bee4e53f56269ac21c8a714d6 WatchSource:0}: Error finding container b479f40556e964a00a8068333cbb0a4e2d81ff5bee4e53f56269ac21c8a714d6: Status 404 returned error can't find the container with id b479f40556e964a00a8068333cbb0a4e2d81ff5bee4e53f56269ac21c8a714d6 Jul 07 13:13:26.883765 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:26.751662 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b8060886d9cb84dfad3d715baebfe6.slice/crio-7afb08f5869cf5d003f5b7a9a5b3e0d72e4ad8a39f9380c7080c4935eef15126 WatchSource:0}: Error finding container 7afb08f5869cf5d003f5b7a9a5b3e0d72e4ad8a39f9380c7080c4935eef15126: Status 404 returned error can't find the container with id 7afb08f5869cf5d003f5b7a9a5b3e0d72e4ad8a39f9380c7080c4935eef15126 Jul 07 13:13:26.883765 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.754864 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" Jul 07 13:13:26.883765 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.755703 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:13:26.883765 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.779352 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 13:13:26.883765 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.780633 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-95.ec2.internal" Jul 07 13:13:26.883765 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.794002 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 13:13:26.883765 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:26.851853 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:13:27.335935 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.335851 2574 apiserver.go:52] "Watching apiserver" Jul 07 13:13:27.347540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.347500 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 13:13:27.349907 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.349871 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-kcfzq","openshift-network-operator/iptables-alerter-r6zsr","openshift-ovn-kubernetes/ovnkube-node-h48b6","kube-system/global-pull-secret-syncer-tq86p","kube-system/kube-apiserver-proxy-ip-10-0-143-95.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp","openshift-cluster-node-tuning-operator/tuned-7h5hc","openshift-image-registry/node-ca-txrsh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal","openshift-multus/multus-additional-cni-plugins-hwk42","openshift-multus/network-metrics-daemon-gcblz","kube-system/konnectivity-agent-zjrkx","openshift-dns/node-resolver-tzl89","openshift-multus/multus-8tndt"] Jul 07 13:13:27.352714 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.352684 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.355020 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.354994 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:27.355181 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.355089 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:27.355274 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.355215 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 13:13:27.356239 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.356212 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:13:27.356239 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.356223 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 13:13:27.356239 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.356231 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m5cx6\"" Jul 07 13:13:27.358043 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.358022 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.360812 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.360643 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:27.360812 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.360761 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:27.361585 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.361377 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 13:13:27.361585 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.361517 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 13:13:27.361585 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.361517 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-drj2w\"" Jul 07 13:13:27.362192 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.362132 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 13:13:27.362476 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.362457 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 13:13:27.362552 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.362475 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 13:13:27.362740 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.362689 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 13:13:27.363704 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.363683 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.365977 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.365956 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 13:13:27.366364 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.366326 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.366471 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.366365 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ct99h\"" Jul 07 13:13:27.366807 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.366770 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 13:13:27.367514 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.367491 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 13:13:27.367867 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.367839 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 13:13:27.369707 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.369684 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 13:13:27.369831 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.369727 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-txrsh" Jul 07 13:13:27.370235 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.370216 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lpvts\"" Jul 07 13:13:27.370586 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.370525 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 13:13:27.372439 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.372413 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 13:13:27.372820 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.372800 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tzl89" Jul 07 13:13:27.373443 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.373412 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 13:13:27.373556 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.373523 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 13:13:27.373697 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.373674 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 13:13:27.373825 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.373805 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jbx84\"" Jul 07 13:13:27.375540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.375514 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r6zsr" Jul 07 13:13:27.376019 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.375706 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 13:13:27.376019 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.375725 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m5j59\"" Jul 07 13:13:27.376019 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.375804 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 13:13:27.376202 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.376135 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 13:13:27.378606 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.378194 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.380206 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.380182 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 13:13:27.380336 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.380230 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:13:27.380952 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.380932 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:27.381043 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.380992 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lqnpt\"" Jul 07 13:13:27.381043 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.381004 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:27.381392 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.381369 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 13:13:27.381622 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.381497 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 13:13:27.381622 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.381577 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 13:13:27.381767 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.381671 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mlhzc\"" Jul 07 13:13:27.382504 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382479 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfs4p\" (UniqueName: \"kubernetes.io/projected/d5eb2177-1423-49f0-b2b1-a583e0646586-kube-api-access-jfs4p\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.382613 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382519 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-sysconfig\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.382613 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382547 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5rsf\" (UniqueName: \"kubernetes.io/projected/fefb5fd3-beb1-426e-b2e0-6defab925a6a-kube-api-access-d5rsf\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:27.382613 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382577 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-run-ovn\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.382613 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382604 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-log-socket\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.382842 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382630 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-run-ovn-kubernetes\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.382842 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382657 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-cni-netd\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.382842 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382681 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-run\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.382842 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382708 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.382842 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382740 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-systemd-units\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.382842 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382765 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-cni-bin\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.382842 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382789 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-modprobe-d\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.382842 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382817 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-tuned\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.383165 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382873 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmsd5\" (UniqueName: \"kubernetes.io/projected/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-kube-api-access-cmsd5\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.383165 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.382932 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-slash\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.383165 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383028 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-etc-openvswitch\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.383165 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383065 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-run-openvswitch\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.383165 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383118 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-sysctl-conf\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.383415 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383168 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-tmp\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.383415 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383202 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-kubernetes\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.383415 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383243 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-run-systemd\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.383415 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383287 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5eb2177-1423-49f0-b2b1-a583e0646586-ovn-node-metrics-cert\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.383415 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383328 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5eb2177-1423-49f0-b2b1-a583e0646586-ovnkube-script-lib\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.383415 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383352 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-systemd\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.383415 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383391 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-lib-modules\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383435 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-var-lib-kubelet\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383467 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-var-lib-openvswitch\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383497 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-node-log\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383528 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-sys\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383561 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-host\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383598 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383611 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383628 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-run-netns\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383664 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5eb2177-1423-49f0-b2b1-a583e0646586-ovnkube-config\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383690 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-sysctl-d\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383727 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-kubelet\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.383764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.383756 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5eb2177-1423-49f0-b2b1-a583e0646586-env-overrides\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.386163 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.386142 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ms4lq\"" Jul 07 13:13:27.386287 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.386188 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 13:13:27.386621 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.386502 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 13:13:27.397122 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.397086 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qtksl" Jul 07 13:13:27.403623 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.403597 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qtksl" Jul 07 13:13:27.431397 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.431332 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" event={"ID":"73b8060886d9cb84dfad3d715baebfe6","Type":"ContainerStarted","Data":"7afb08f5869cf5d003f5b7a9a5b3e0d72e4ad8a39f9380c7080c4935eef15126"} Jul 07 13:13:27.433838 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.433795 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-95.ec2.internal" event={"ID":"18e3cb94201ad665b039cc13756723f0","Type":"ContainerStarted","Data":"b479f40556e964a00a8068333cbb0a4e2d81ff5bee4e53f56269ac21c8a714d6"} Jul 07 13:13:27.456794 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.456760 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 13:13:27.484015 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.483976 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-sysctl-conf\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484015 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484015 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-tmp\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484238 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484044 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.484238 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484063 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-kubernetes\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484238 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484081 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-run-systemd\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.484238 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484101 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-lib-modules\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484238 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484126 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-var-lib-kubelet\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484238 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484153 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-var-lib-kubelet\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.484238 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484180 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bdead414-6f59-4be1-8d50-131a7d61342d-multus-daemon-config\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.484238 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484206 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5wgf\" (UniqueName: \"kubernetes.io/projected/bdead414-6f59-4be1-8d50-131a7d61342d-kube-api-access-k5wgf\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.484238 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484207 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-run-systemd\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.484238 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484231 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-device-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484256 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-node-log\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484293 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-kubernetes\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484209 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-sysctl-conf\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484298 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-sys\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484339 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-host\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484364 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-var-lib-kubelet\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484366 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7626\" (UniqueName: \"kubernetes.io/projected/78b4321d-698d-48e8-b2d7-be1e8f43ccee-kube-api-access-c7626\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484401 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-lib-modules\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484405 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crh8h\" (UniqueName: \"kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h\") pod \"network-check-target-kcfzq\" (UID: \"49219a1b-1827-48a1-96c0-35b6b6695373\") " pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484425 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-node-log\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484450 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff29c277-a852-463a-8e9f-e066329a2095-konnectivity-ca\") pod \"konnectivity-agent-zjrkx\" (UID: \"ff29c277-a852-463a-8e9f-e066329a2095\") " pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484483 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-registration-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484488 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-sys\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.484672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484525 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-sysctl-d\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484527 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-host\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484514 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484554 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f3c64c-d89e-4583-955e-66d1c534336d-tmp-dir\") pod \"node-resolver-tzl89\" (UID: \"c9f3c64c-d89e-4583-955e-66d1c534336d\") " pod="openshift-dns/node-resolver-tzl89" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484580 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/062eca81-d98c-4875-b05a-783b32893f22-host-slash\") pod \"iptables-alerter-r6zsr\" (UID: \"062eca81-d98c-4875-b05a-783b32893f22\") " pod="openshift-network-operator/iptables-alerter-r6zsr" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484640 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff29c277-a852-463a-8e9f-e066329a2095-agent-certs\") pod \"konnectivity-agent-zjrkx\" (UID: \"ff29c277-a852-463a-8e9f-e066329a2095\") " pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484671 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2bb5f676-7523-4cab-b7de-fd7cbd99581d-kubelet-config\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484682 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-sysctl-d\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484714 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484753 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484788 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5eb2177-1423-49f0-b2b1-a583e0646586-env-overrides\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484815 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3829f252-2935-4883-9ce3-b3823a2c3572-host\") pod \"node-ca-txrsh\" (UID: \"3829f252-2935-4883-9ce3-b3823a2c3572\") " pod="openshift-image-registry/node-ca-txrsh" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484840 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-hostroot\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484864 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-sysconfig\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.485460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484896 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-run-ovn\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484928 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-system-cni-dir\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484942 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-sysconfig\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484956 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/78b4321d-698d-48e8-b2d7-be1e8f43ccee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.484998 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-run-netns\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485018 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-run-ovn\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485026 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-run-multus-certs\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485066 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485095 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-cnibin\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485140 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-sys-fs\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485170 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-cni-bin\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485183 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485195 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-modprobe-d\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485224 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmsd5\" (UniqueName: \"kubernetes.io/projected/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-kube-api-access-cmsd5\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.486271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485233 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-cni-bin\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485251 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrcft\" (UniqueName: \"kubernetes.io/projected/3829f252-2935-4883-9ce3-b3823a2c3572-kube-api-access-lrcft\") pod \"node-ca-txrsh\" (UID: \"3829f252-2935-4883-9ce3-b3823a2c3572\") " pod="openshift-image-registry/node-ca-txrsh" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485294 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78b4321d-698d-48e8-b2d7-be1e8f43ccee-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485322 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-modprobe-d\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485322 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-etc-openvswitch\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485371 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-etc-openvswitch\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485375 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-multus-conf-dir\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485407 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5eb2177-1423-49f0-b2b1-a583e0646586-ovn-node-metrics-cert\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485436 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5eb2177-1423-49f0-b2b1-a583e0646586-ovnkube-script-lib\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485405 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5eb2177-1423-49f0-b2b1-a583e0646586-env-overrides\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485462 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-systemd\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485489 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7cjx\" (UniqueName: \"kubernetes.io/projected/c9f3c64c-d89e-4583-955e-66d1c534336d-kube-api-access-r7cjx\") pod \"node-resolver-tzl89\" (UID: \"c9f3c64c-d89e-4583-955e-66d1c534336d\") " pod="openshift-dns/node-resolver-tzl89" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485517 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/062eca81-d98c-4875-b05a-783b32893f22-iptables-alerter-script\") pod \"iptables-alerter-r6zsr\" (UID: \"062eca81-d98c-4875-b05a-783b32893f22\") " pod="openshift-network-operator/iptables-alerter-r6zsr" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485530 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-systemd\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.487027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485560 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-var-lib-openvswitch\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485587 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-var-lib-openvswitch\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485584 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3829f252-2935-4883-9ce3-b3823a2c3572-serviceca\") pod \"node-ca-txrsh\" (UID: \"3829f252-2935-4883-9ce3-b3823a2c3572\") " pod="openshift-image-registry/node-ca-txrsh" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485619 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdead414-6f59-4be1-8d50-131a7d61342d-cni-binary-copy\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485662 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-var-lib-cni-bin\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485701 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-etc-selinux\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485731 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485758 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-run-netns\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485782 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5eb2177-1423-49f0-b2b1-a583e0646586-ovnkube-config\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485807 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-cnibin\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485834 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-os-release\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.485836 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485873 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-multus-socket-dir-parent\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485843 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-run-netns\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.487766 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.485914 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs podName:fefb5fd3-beb1-426e-b2e0-6defab925a6a nodeName:}" failed. No retries permitted until 2026-07-07 13:13:27.985891 +0000 UTC m=+3.040312660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs") pod "network-metrics-daemon-gcblz" (UID: "fefb5fd3-beb1-426e-b2e0-6defab925a6a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485933 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgvs5\" (UniqueName: \"kubernetes.io/projected/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-kube-api-access-qgvs5\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485964 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-kubelet\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.485988 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-os-release\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486013 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-socket-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486040 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfs4p\" (UniqueName: \"kubernetes.io/projected/d5eb2177-1423-49f0-b2b1-a583e0646586-kube-api-access-jfs4p\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486065 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5rsf\" (UniqueName: \"kubernetes.io/projected/fefb5fd3-beb1-426e-b2e0-6defab925a6a-kube-api-access-d5rsf\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486069 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5eb2177-1423-49f0-b2b1-a583e0646586-ovnkube-script-lib\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486089 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-log-socket\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486117 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-run-ovn-kubernetes\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486156 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-kubelet\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486181 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-run-ovn-kubernetes\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486213 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-cni-netd\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486237 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-log-socket\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.488458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486243 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-run\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486284 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/78b4321d-698d-48e8-b2d7-be1e8f43ccee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486315 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-multus-cni-dir\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486357 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-run\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486361 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-var-lib-cni-multus\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486364 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5eb2177-1423-49f0-b2b1-a583e0646586-ovnkube-config\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486386 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-etc-kubernetes\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486399 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-cni-netd\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486410 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9f3c64c-d89e-4583-955e-66d1c534336d-hosts-file\") pod \"node-resolver-tzl89\" (UID: \"c9f3c64c-d89e-4583-955e-66d1c534336d\") " pod="openshift-dns/node-resolver-tzl89" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486472 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2bb5f676-7523-4cab-b7de-fd7cbd99581d-dbus\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486499 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-systemd-units\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486543 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-tuned\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486579 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-system-cni-dir\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486552 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-systemd-units\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.489081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486605 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-run-k8s-cni-cncf-io\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.489569 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486632 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwxvc\" (UniqueName: \"kubernetes.io/projected/062eca81-d98c-4875-b05a-783b32893f22-kube-api-access-rwxvc\") pod \"iptables-alerter-r6zsr\" (UID: \"062eca81-d98c-4875-b05a-783b32893f22\") " pod="openshift-network-operator/iptables-alerter-r6zsr" Jul 07 13:13:27.489569 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486662 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-slash\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.489569 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486711 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-run-openvswitch\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.489569 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486795 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-host-slash\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.489569 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.486845 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5eb2177-1423-49f0-b2b1-a583e0646586-run-openvswitch\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.489569 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.488744 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-tmp\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.489569 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.488959 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-etc-tuned\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.489569 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.488959 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5eb2177-1423-49f0-b2b1-a583e0646586-ovn-node-metrics-cert\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.496086 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.496054 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmsd5\" (UniqueName: \"kubernetes.io/projected/72ecc126-5631-4fa3-a4bd-4e636dd7e04f-kube-api-access-cmsd5\") pod \"tuned-7h5hc\" (UID: \"72ecc126-5631-4fa3-a4bd-4e636dd7e04f\") " pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.496242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.496222 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfs4p\" (UniqueName: \"kubernetes.io/projected/d5eb2177-1423-49f0-b2b1-a583e0646586-kube-api-access-jfs4p\") pod \"ovnkube-node-h48b6\" (UID: \"d5eb2177-1423-49f0-b2b1-a583e0646586\") " pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.496468 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.496442 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5rsf\" (UniqueName: \"kubernetes.io/projected/fefb5fd3-beb1-426e-b2e0-6defab925a6a-kube-api-access-d5rsf\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:27.580935 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.580897 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:13:27.587657 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.587510 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7cjx\" (UniqueName: \"kubernetes.io/projected/c9f3c64c-d89e-4583-955e-66d1c534336d-kube-api-access-r7cjx\") pod \"node-resolver-tzl89\" (UID: \"c9f3c64c-d89e-4583-955e-66d1c534336d\") " pod="openshift-dns/node-resolver-tzl89" Jul 07 13:13:27.587657 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.587566 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/062eca81-d98c-4875-b05a-783b32893f22-iptables-alerter-script\") pod \"iptables-alerter-r6zsr\" (UID: \"062eca81-d98c-4875-b05a-783b32893f22\") " pod="openshift-network-operator/iptables-alerter-r6zsr" Jul 07 13:13:27.587905 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.587879 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3829f252-2935-4883-9ce3-b3823a2c3572-serviceca\") pod \"node-ca-txrsh\" (UID: \"3829f252-2935-4883-9ce3-b3823a2c3572\") " pod="openshift-image-registry/node-ca-txrsh" Jul 07 13:13:27.587983 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.587922 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdead414-6f59-4be1-8d50-131a7d61342d-cni-binary-copy\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.587983 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.587952 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-var-lib-cni-bin\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.587983 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.587978 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-etc-selinux\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.588163 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588022 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-cnibin\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.588163 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588050 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-os-release\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.588163 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588074 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-multus-socket-dir-parent\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.588163 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588101 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvs5\" (UniqueName: \"kubernetes.io/projected/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-kube-api-access-qgvs5\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.588163 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588127 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-os-release\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.588163 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588151 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-socket-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.588498 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588184 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/78b4321d-698d-48e8-b2d7-be1e8f43ccee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.588498 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588225 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-multus-cni-dir\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.588498 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588253 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-var-lib-cni-multus\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.588498 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588274 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/062eca81-d98c-4875-b05a-783b32893f22-iptables-alerter-script\") pod \"iptables-alerter-r6zsr\" (UID: \"062eca81-d98c-4875-b05a-783b32893f22\") " pod="openshift-network-operator/iptables-alerter-r6zsr" Jul 07 13:13:27.588498 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588278 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-etc-kubernetes\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.588498 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588348 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-etc-kubernetes\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.588498 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9f3c64c-d89e-4583-955e-66d1c534336d-hosts-file\") pod \"node-resolver-tzl89\" (UID: \"c9f3c64c-d89e-4583-955e-66d1c534336d\") " pod="openshift-dns/node-resolver-tzl89" Jul 07 13:13:27.588498 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588469 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2bb5f676-7523-4cab-b7de-fd7cbd99581d-dbus\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588503 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-system-cni-dir\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588535 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-run-k8s-cni-cncf-io\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588563 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwxvc\" (UniqueName: \"kubernetes.io/projected/062eca81-d98c-4875-b05a-783b32893f22-kube-api-access-rwxvc\") pod \"iptables-alerter-r6zsr\" (UID: \"062eca81-d98c-4875-b05a-783b32893f22\") " pod="openshift-network-operator/iptables-alerter-r6zsr" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588598 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588630 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-var-lib-kubelet\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588661 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bdead414-6f59-4be1-8d50-131a7d61342d-multus-daemon-config\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588687 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5wgf\" (UniqueName: \"kubernetes.io/projected/bdead414-6f59-4be1-8d50-131a7d61342d-kube-api-access-k5wgf\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588714 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-device-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588745 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7626\" (UniqueName: \"kubernetes.io/projected/78b4321d-698d-48e8-b2d7-be1e8f43ccee-kube-api-access-c7626\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588773 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crh8h\" (UniqueName: \"kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h\") pod \"network-check-target-kcfzq\" (UID: \"49219a1b-1827-48a1-96c0-35b6b6695373\") " pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588800 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff29c277-a852-463a-8e9f-e066329a2095-konnectivity-ca\") pod \"konnectivity-agent-zjrkx\" (UID: \"ff29c277-a852-463a-8e9f-e066329a2095\") " pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588812 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3829f252-2935-4883-9ce3-b3823a2c3572-serviceca\") pod \"node-ca-txrsh\" (UID: \"3829f252-2935-4883-9ce3-b3823a2c3572\") " pod="openshift-image-registry/node-ca-txrsh" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588829 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-registration-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.588909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588877 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f3c64c-d89e-4583-955e-66d1c534336d-tmp-dir\") pod \"node-resolver-tzl89\" (UID: \"c9f3c64c-d89e-4583-955e-66d1c534336d\") " pod="openshift-dns/node-resolver-tzl89" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/062eca81-d98c-4875-b05a-783b32893f22-host-slash\") pod \"iptables-alerter-r6zsr\" (UID: \"062eca81-d98c-4875-b05a-783b32893f22\") " pod="openshift-network-operator/iptables-alerter-r6zsr" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588931 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff29c277-a852-463a-8e9f-e066329a2095-agent-certs\") pod \"konnectivity-agent-zjrkx\" (UID: \"ff29c277-a852-463a-8e9f-e066329a2095\") " pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588956 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2bb5f676-7523-4cab-b7de-fd7cbd99581d-kubelet-config\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.588986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589012 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589045 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3829f252-2935-4883-9ce3-b3823a2c3572-host\") pod \"node-ca-txrsh\" (UID: \"3829f252-2935-4883-9ce3-b3823a2c3572\") " pod="openshift-image-registry/node-ca-txrsh" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589060 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-run-k8s-cni-cncf-io\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589075 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-hostroot\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589120 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-system-cni-dir\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589129 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-hostroot\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589165 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/78b4321d-698d-48e8-b2d7-be1e8f43ccee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589196 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-run-netns\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589217 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9f3c64c-d89e-4583-955e-66d1c534336d-hosts-file\") pod \"node-resolver-tzl89\" (UID: \"c9f3c64c-d89e-4583-955e-66d1c534336d\") " pod="openshift-dns/node-resolver-tzl89" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589226 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-run-multus-certs\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.589630 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589237 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-var-lib-kubelet\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589280 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-run-multus-certs\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589307 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-cnibin\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589339 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-sys-fs\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589347 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2bb5f676-7523-4cab-b7de-fd7cbd99581d-kubelet-config\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589349 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-system-cni-dir\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589354 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdead414-6f59-4be1-8d50-131a7d61342d-cni-binary-copy\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589352 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-var-lib-cni-bin\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589376 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrcft\" (UniqueName: \"kubernetes.io/projected/3829f252-2935-4883-9ce3-b3823a2c3572-kube-api-access-lrcft\") pod \"node-ca-txrsh\" (UID: \"3829f252-2935-4883-9ce3-b3823a2c3572\") " pod="openshift-image-registry/node-ca-txrsh" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589397 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78b4321d-698d-48e8-b2d7-be1e8f43ccee-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589413 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-sys-fs\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-multus-conf-dir\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589457 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-cnibin\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589475 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-etc-selinux\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589486 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-socket-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.590427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589504 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2bb5f676-7523-4cab-b7de-fd7cbd99581d-dbus\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589505 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-multus-cni-dir\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.589530 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589538 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-var-lib-cni-multus\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.589583 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret podName:2bb5f676-7523-4cab-b7de-fd7cbd99581d nodeName:}" failed. No retries permitted until 2026-07-07 13:13:28.089568532 +0000 UTC m=+3.143990169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret") pod "global-pull-secret-syncer-tq86p" (UID: "2bb5f676-7523-4cab-b7de-fd7cbd99581d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589616 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589671 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-os-release\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589676 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-device-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589693 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-os-release\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589770 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-cnibin\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589822 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-multus-conf-dir\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589169 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78b4321d-698d-48e8-b2d7-be1e8f43ccee-system-cni-dir\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589698 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589940 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-host-run-netns\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.591190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589988 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3829f252-2935-4883-9ce3-b3823a2c3572-host\") pod \"node-ca-txrsh\" (UID: \"3829f252-2935-4883-9ce3-b3823a2c3572\") " pod="openshift-image-registry/node-ca-txrsh" Jul 07 13:13:27.591936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.589987 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bdead414-6f59-4be1-8d50-131a7d61342d-multus-socket-dir-parent\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.591936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.590054 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-registration-dir\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.591936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.590107 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/062eca81-d98c-4875-b05a-783b32893f22-host-slash\") pod \"iptables-alerter-r6zsr\" (UID: \"062eca81-d98c-4875-b05a-783b32893f22\") " pod="openshift-network-operator/iptables-alerter-r6zsr" Jul 07 13:13:27.591936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.590153 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff29c277-a852-463a-8e9f-e066329a2095-konnectivity-ca\") pod \"konnectivity-agent-zjrkx\" (UID: \"ff29c277-a852-463a-8e9f-e066329a2095\") " pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:27.591936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.590304 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f3c64c-d89e-4583-955e-66d1c534336d-tmp-dir\") pod \"node-resolver-tzl89\" (UID: \"c9f3c64c-d89e-4583-955e-66d1c534336d\") " pod="openshift-dns/node-resolver-tzl89" Jul 07 13:13:27.591936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.590317 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bdead414-6f59-4be1-8d50-131a7d61342d-multus-daemon-config\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.591936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.590892 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78b4321d-698d-48e8-b2d7-be1e8f43ccee-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.591936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.590941 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/78b4321d-698d-48e8-b2d7-be1e8f43ccee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.591936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.590985 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/78b4321d-698d-48e8-b2d7-be1e8f43ccee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.594292 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.594260 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff29c277-a852-463a-8e9f-e066329a2095-agent-certs\") pod \"konnectivity-agent-zjrkx\" (UID: \"ff29c277-a852-463a-8e9f-e066329a2095\") " pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:27.598674 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.598604 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7cjx\" (UniqueName: \"kubernetes.io/projected/c9f3c64c-d89e-4583-955e-66d1c534336d-kube-api-access-r7cjx\") pod \"node-resolver-tzl89\" (UID: \"c9f3c64c-d89e-4583-955e-66d1c534336d\") " pod="openshift-dns/node-resolver-tzl89" Jul 07 13:13:27.599761 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.599429 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:13:27.599761 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.599454 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:13:27.599761 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.599468 2574 projected.go:196] Error preparing data for projected volume kube-api-access-crh8h for pod openshift-network-diagnostics/network-check-target-kcfzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:27.599761 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.599616 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h podName:49219a1b-1827-48a1-96c0-35b6b6695373 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:28.099516274 +0000 UTC m=+3.153937931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-crh8h" (UniqueName: "kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h") pod "network-check-target-kcfzq" (UID: "49219a1b-1827-48a1-96c0-35b6b6695373") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:27.603346 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.603299 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrcft\" (UniqueName: \"kubernetes.io/projected/3829f252-2935-4883-9ce3-b3823a2c3572-kube-api-access-lrcft\") pod \"node-ca-txrsh\" (UID: \"3829f252-2935-4883-9ce3-b3823a2c3572\") " pod="openshift-image-registry/node-ca-txrsh" Jul 07 13:13:27.605691 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.605664 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7626\" (UniqueName: \"kubernetes.io/projected/78b4321d-698d-48e8-b2d7-be1e8f43ccee-kube-api-access-c7626\") pod \"multus-additional-cni-plugins-hwk42\" (UID: \"78b4321d-698d-48e8-b2d7-be1e8f43ccee\") " pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.605805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.605730 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwxvc\" (UniqueName: \"kubernetes.io/projected/062eca81-d98c-4875-b05a-783b32893f22-kube-api-access-rwxvc\") pod \"iptables-alerter-r6zsr\" (UID: \"062eca81-d98c-4875-b05a-783b32893f22\") " pod="openshift-network-operator/iptables-alerter-r6zsr" Jul 07 13:13:27.607427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.607401 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgvs5\" (UniqueName: \"kubernetes.io/projected/34ba3f19-8b1e-4d3d-904c-ef07ce4f5502-kube-api-access-qgvs5\") pod \"aws-ebs-csi-driver-node-kkkzp\" (UID: \"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.607655 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.607633 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5wgf\" (UniqueName: \"kubernetes.io/projected/bdead414-6f59-4be1-8d50-131a7d61342d-kube-api-access-k5wgf\") pod \"multus-8tndt\" (UID: \"bdead414-6f59-4be1-8d50-131a7d61342d\") " pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.666798 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.666756 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" Jul 07 13:13:27.678987 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.678751 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:27.681105 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:27.681054 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ecc126_5631_4fa3_a4bd_4e636dd7e04f.slice/crio-cf25b71fc497c2f54ab819a53da9fbf2972d0e5005a845357e8576e9493be0b7 WatchSource:0}: Error finding container cf25b71fc497c2f54ab819a53da9fbf2972d0e5005a845357e8576e9493be0b7: Status 404 returned error can't find the container with id cf25b71fc497c2f54ab819a53da9fbf2972d0e5005a845357e8576e9493be0b7 Jul 07 13:13:27.690000 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.689764 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-txrsh" Jul 07 13:13:27.690906 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:27.690870 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5eb2177_1423_49f0_b2b1_a583e0646586.slice/crio-7a2117187b1666d57574dbf6c16521718054bb68f497188f4aaa0cce457fca0f WatchSource:0}: Error finding container 7a2117187b1666d57574dbf6c16521718054bb68f497188f4aaa0cce457fca0f: Status 404 returned error can't find the container with id 7a2117187b1666d57574dbf6c16521718054bb68f497188f4aaa0cce457fca0f Jul 07 13:13:27.695420 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.695336 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8tndt" Jul 07 13:13:27.703878 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.703846 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" Jul 07 13:13:27.704645 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:27.704608 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3829f252_2935_4883_9ce3_b3823a2c3572.slice/crio-e920c10885e370cf7a4fe7a01fc11babba8a381a21cd912f762a96465d2c38c1 WatchSource:0}: Error finding container e920c10885e370cf7a4fe7a01fc11babba8a381a21cd912f762a96465d2c38c1: Status 404 returned error can't find the container with id e920c10885e370cf7a4fe7a01fc11babba8a381a21cd912f762a96465d2c38c1 Jul 07 13:13:27.710215 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:27.710181 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdead414_6f59_4be1_8d50_131a7d61342d.slice/crio-987e83e8ee547bd54d02e42ceefff25372458c1142c26745320e14d43a994407 WatchSource:0}: Error finding container 987e83e8ee547bd54d02e42ceefff25372458c1142c26745320e14d43a994407: Status 404 returned error can't find the container with id 987e83e8ee547bd54d02e42ceefff25372458c1142c26745320e14d43a994407 Jul 07 13:13:27.711410 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.711204 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tzl89" Jul 07 13:13:27.720761 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:27.720725 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ba3f19_8b1e_4d3d_904c_ef07ce4f5502.slice/crio-b08f9b1424004f35e8ed4534e77057eb863f2ccfede3aee9596823df82797f1b WatchSource:0}: Error finding container b08f9b1424004f35e8ed4534e77057eb863f2ccfede3aee9596823df82797f1b: Status 404 returned error can't find the container with id b08f9b1424004f35e8ed4534e77057eb863f2ccfede3aee9596823df82797f1b Jul 07 13:13:27.720761 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.720742 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r6zsr" Jul 07 13:13:27.730456 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.730410 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hwk42" Jul 07 13:13:27.736800 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.736769 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:27.738709 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:27.738673 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod062eca81_d98c_4875_b05a_783b32893f22.slice/crio-2074f6ced21cd4eb5f141506fa5c3a15e95540a6133eef7c09fa41b575c47797 WatchSource:0}: Error finding container 2074f6ced21cd4eb5f141506fa5c3a15e95540a6133eef7c09fa41b575c47797: Status 404 returned error can't find the container with id 2074f6ced21cd4eb5f141506fa5c3a15e95540a6133eef7c09fa41b575c47797 Jul 07 13:13:27.746500 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:27.746464 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78b4321d_698d_48e8_b2d7_be1e8f43ccee.slice/crio-8f4bf3f5106a316e3f5f006a6d4b357af5cb141bc88200b87a9465c626cc2ced WatchSource:0}: Error finding container 8f4bf3f5106a316e3f5f006a6d4b357af5cb141bc88200b87a9465c626cc2ced: Status 404 returned error can't find the container with id 8f4bf3f5106a316e3f5f006a6d4b357af5cb141bc88200b87a9465c626cc2ced Jul 07 13:13:27.751250 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:13:27.751208 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff29c277_a852_463a_8e9f_e066329a2095.slice/crio-853e42e2b44fb1445b118b272d3b9f4add4ed6e19843bbb3905d2d9c13876dca WatchSource:0}: Error finding container 853e42e2b44fb1445b118b272d3b9f4add4ed6e19843bbb3905d2d9c13876dca: Status 404 returned error can't find the container with id 853e42e2b44fb1445b118b272d3b9f4add4ed6e19843bbb3905d2d9c13876dca Jul 07 13:13:27.993182 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:27.993077 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:27.993374 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.993308 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:27.993429 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:27.993395 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs podName:fefb5fd3-beb1-426e-b2e0-6defab925a6a nodeName:}" failed. No retries permitted until 2026-07-07 13:13:28.993372359 +0000 UTC m=+4.047794002 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs") pod "network-metrics-daemon-gcblz" (UID: "fefb5fd3-beb1-426e-b2e0-6defab925a6a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:28.094466 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.094429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:28.094639 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:28.094572 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:28.094714 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:28.094643 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret podName:2bb5f676-7523-4cab-b7de-fd7cbd99581d nodeName:}" failed. No retries permitted until 2026-07-07 13:13:29.094622205 +0000 UTC m=+4.149043846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret") pod "global-pull-secret-syncer-tq86p" (UID: "2bb5f676-7523-4cab-b7de-fd7cbd99581d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:28.194878 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.194837 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crh8h\" (UniqueName: \"kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h\") pod \"network-check-target-kcfzq\" (UID: \"49219a1b-1827-48a1-96c0-35b6b6695373\") " pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:28.195094 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:28.195069 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:13:28.195094 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:28.195098 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:13:28.195344 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:28.195112 2574 projected.go:196] Error preparing data for projected volume kube-api-access-crh8h for pod openshift-network-diagnostics/network-check-target-kcfzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:28.195344 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:28.195180 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h podName:49219a1b-1827-48a1-96c0-35b6b6695373 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:29.19516008 +0000 UTC m=+4.249581732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-crh8h" (UniqueName: "kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h") pod "network-check-target-kcfzq" (UID: "49219a1b-1827-48a1-96c0-35b6b6695373") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:28.393635 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:28.393514 2574 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:28389dd118629aa9d7c2525202f3ad6359afb98b6a3bf7672cb1ecfae2418e23" Jul 07 13:13:28.393818 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:28.393674 2574 kuberuntime_manager.go:1459] "Unhandled Error" err="container dns-node-resolver start failed in pod node-resolver-tzl89_openshift-dns(c9f3c64c-d89e-4583-955e-66d1c534336d): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 07 13:13:28.393818 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:28.393761 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-dns/node-resolver-tzl89" podUID="c9f3c64c-d89e-4583-955e-66d1c534336d" Jul 07 13:13:28.407440 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.407366 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 13:08:27 +0000 UTC" deadline="2028-03-13 14:52:50.963903571 +0000 UTC" Jul 07 13:13:28.407440 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.407425 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14761h39m22.556481523s" Jul 07 13:13:28.426044 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.426002 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:28.426231 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:28.426130 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:28.437840 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.437796 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwk42" event={"ID":"78b4321d-698d-48e8-b2d7-be1e8f43ccee","Type":"ContainerStarted","Data":"8f4bf3f5106a316e3f5f006a6d4b357af5cb141bc88200b87a9465c626cc2ced"} Jul 07 13:13:28.439171 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.439140 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r6zsr" event={"ID":"062eca81-d98c-4875-b05a-783b32893f22","Type":"ContainerStarted","Data":"2074f6ced21cd4eb5f141506fa5c3a15e95540a6133eef7c09fa41b575c47797"} Jul 07 13:13:28.440555 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.440517 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" event={"ID":"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502","Type":"ContainerStarted","Data":"b08f9b1424004f35e8ed4534e77057eb863f2ccfede3aee9596823df82797f1b"} Jul 07 13:13:28.441907 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.441878 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8tndt" event={"ID":"bdead414-6f59-4be1-8d50-131a7d61342d","Type":"ContainerStarted","Data":"987e83e8ee547bd54d02e42ceefff25372458c1142c26745320e14d43a994407"} Jul 07 13:13:28.443248 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.443214 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" event={"ID":"d5eb2177-1423-49f0-b2b1-a583e0646586","Type":"ContainerStarted","Data":"7a2117187b1666d57574dbf6c16521718054bb68f497188f4aaa0cce457fca0f"} Jul 07 13:13:28.445014 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.444979 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" event={"ID":"72ecc126-5631-4fa3-a4bd-4e636dd7e04f","Type":"ContainerStarted","Data":"cf25b71fc497c2f54ab819a53da9fbf2972d0e5005a845357e8576e9493be0b7"} Jul 07 13:13:28.446193 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.446160 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zjrkx" event={"ID":"ff29c277-a852-463a-8e9f-e066329a2095","Type":"ContainerStarted","Data":"853e42e2b44fb1445b118b272d3b9f4add4ed6e19843bbb3905d2d9c13876dca"} Jul 07 13:13:28.447382 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.447358 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tzl89" event={"ID":"c9f3c64c-d89e-4583-955e-66d1c534336d","Type":"ContainerStarted","Data":"f38984ab02a6d2934cc4d294fc82640a18737af0c64c65e8dda97b71cba21d17"} Jul 07 13:13:28.448579 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:28.448553 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-txrsh" event={"ID":"3829f252-2935-4883-9ce3-b3823a2c3572","Type":"ContainerStarted","Data":"e920c10885e370cf7a4fe7a01fc11babba8a381a21cd912f762a96465d2c38c1"} Jul 07 13:13:28.448671 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:28.448645 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:28389dd118629aa9d7c2525202f3ad6359afb98b6a3bf7672cb1ecfae2418e23\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-dns/node-resolver-tzl89" podUID="c9f3c64c-d89e-4583-955e-66d1c534336d" Jul 07 13:13:29.002023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:29.001976 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:29.002839 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:29.002143 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:29.002839 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:29.002212 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs podName:fefb5fd3-beb1-426e-b2e0-6defab925a6a nodeName:}" failed. No retries permitted until 2026-07-07 13:13:31.002190605 +0000 UTC m=+6.056612244 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs") pod "network-metrics-daemon-gcblz" (UID: "fefb5fd3-beb1-426e-b2e0-6defab925a6a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:29.103192 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:29.103142 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:29.103424 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:29.103358 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:29.103505 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:29.103427 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret podName:2bb5f676-7523-4cab-b7de-fd7cbd99581d nodeName:}" failed. No retries permitted until 2026-07-07 13:13:31.103407915 +0000 UTC m=+6.157829568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret") pod "global-pull-secret-syncer-tq86p" (UID: "2bb5f676-7523-4cab-b7de-fd7cbd99581d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:29.204448 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:29.204388 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crh8h\" (UniqueName: \"kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h\") pod \"network-check-target-kcfzq\" (UID: \"49219a1b-1827-48a1-96c0-35b6b6695373\") " pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:29.204626 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:29.204566 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:13:29.204626 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:29.204589 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:13:29.204626 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:29.204602 2574 projected.go:196] Error preparing data for projected volume kube-api-access-crh8h for pod openshift-network-diagnostics/network-check-target-kcfzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:29.204796 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:29.204667 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h podName:49219a1b-1827-48a1-96c0-35b6b6695373 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:31.204644122 +0000 UTC m=+6.259065781 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-crh8h" (UniqueName: "kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h") pod "network-check-target-kcfzq" (UID: "49219a1b-1827-48a1-96c0-35b6b6695373") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:29.408685 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:29.408617 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 13:08:27 +0000 UTC" deadline="2027-12-22 20:34:00.914874335 +0000 UTC" Jul 07 13:13:29.408685 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:29.408658 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12799h20m31.506220911s" Jul 07 13:13:29.436410 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:29.427671 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:29.436410 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:29.428058 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:29.436410 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:29.428393 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:29.436410 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:29.428510 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:29.465191 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:29.465147 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-95.ec2.internal" event={"ID":"18e3cb94201ad665b039cc13756723f0","Type":"ContainerStarted","Data":"b66d2aeccdf979a801a1bed30dd279cbb56f9030fac57fdca7ba17d46371c342"} Jul 07 13:13:29.484542 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:29.484458 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-95.ec2.internal" podStartSLOduration=2.484434349 podStartE2EDuration="2.484434349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:29.484210629 +0000 UTC m=+4.538632295" watchObservedRunningTime="2026-07-07 13:13:29.484434349 +0000 UTC m=+4.538856013" Jul 07 13:13:30.426863 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:30.426286 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:30.426863 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:30.426453 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:30.483899 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:30.483804 2574 generic.go:356] "Generic (PLEG): container finished" podID="73b8060886d9cb84dfad3d715baebfe6" containerID="8f72b680ad6bd07a4ed10f7e1931cdef51f2960b13f6d927d6ab481e80ac31d4" exitCode=0 Jul 07 13:13:30.485095 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:30.485056 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" event={"ID":"73b8060886d9cb84dfad3d715baebfe6","Type":"ContainerDied","Data":"8f72b680ad6bd07a4ed10f7e1931cdef51f2960b13f6d927d6ab481e80ac31d4"} Jul 07 13:13:31.028805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:31.028692 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:31.029014 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:31.028921 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:31.029014 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:31.028999 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs podName:fefb5fd3-beb1-426e-b2e0-6defab925a6a nodeName:}" failed. No retries permitted until 2026-07-07 13:13:35.028977001 +0000 UTC m=+10.083398646 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs") pod "network-metrics-daemon-gcblz" (UID: "fefb5fd3-beb1-426e-b2e0-6defab925a6a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:31.130220 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:31.130170 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:31.130440 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:31.130398 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:31.130513 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:31.130466 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret podName:2bb5f676-7523-4cab-b7de-fd7cbd99581d nodeName:}" failed. No retries permitted until 2026-07-07 13:13:35.130444789 +0000 UTC m=+10.184866432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret") pod "global-pull-secret-syncer-tq86p" (UID: "2bb5f676-7523-4cab-b7de-fd7cbd99581d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:31.231355 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:31.231254 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crh8h\" (UniqueName: \"kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h\") pod \"network-check-target-kcfzq\" (UID: \"49219a1b-1827-48a1-96c0-35b6b6695373\") " pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:31.231567 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:31.231490 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:13:31.231567 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:31.231511 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:13:31.231567 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:31.231523 2574 projected.go:196] Error preparing data for projected volume kube-api-access-crh8h for pod openshift-network-diagnostics/network-check-target-kcfzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:31.231741 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:31.231586 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h podName:49219a1b-1827-48a1-96c0-35b6b6695373 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:35.231565356 +0000 UTC m=+10.285987008 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-crh8h" (UniqueName: "kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h") pod "network-check-target-kcfzq" (UID: "49219a1b-1827-48a1-96c0-35b6b6695373") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:31.428566 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:31.428466 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:31.429100 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:31.428631 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:31.429251 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:31.429226 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:31.429385 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:31.429361 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:31.496489 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:31.495649 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" event={"ID":"73b8060886d9cb84dfad3d715baebfe6","Type":"ContainerStarted","Data":"732d670145863aa363dd0e098da005232ea9407e47d3565ba0be7343875692b6"} Jul 07 13:13:31.517057 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:31.516871 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-95.ec2.internal" podStartSLOduration=4.516844855 podStartE2EDuration="4.516844855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:31.515621397 +0000 UTC m=+6.570043062" watchObservedRunningTime="2026-07-07 13:13:31.516844855 +0000 UTC m=+6.571266519" Jul 07 13:13:32.426401 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:32.426357 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:32.426608 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:32.426543 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:33.426640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:33.426054 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:33.426640 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:33.426214 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:33.426640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:33.426278 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:33.426640 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:33.426400 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:34.426339 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:34.426228 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:34.426524 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:34.426402 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:35.064584 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:35.064524 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:35.065075 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:35.064702 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:35.065075 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:35.064780 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs podName:fefb5fd3-beb1-426e-b2e0-6defab925a6a nodeName:}" failed. No retries permitted until 2026-07-07 13:13:43.064758952 +0000 UTC m=+18.119180587 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs") pod "network-metrics-daemon-gcblz" (UID: "fefb5fd3-beb1-426e-b2e0-6defab925a6a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:35.165608 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:35.165568 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:35.165803 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:35.165757 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:35.165874 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:35.165821 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret podName:2bb5f676-7523-4cab-b7de-fd7cbd99581d nodeName:}" failed. No retries permitted until 2026-07-07 13:13:43.165802639 +0000 UTC m=+18.220224276 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret") pod "global-pull-secret-syncer-tq86p" (UID: "2bb5f676-7523-4cab-b7de-fd7cbd99581d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:35.266544 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:35.266503 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crh8h\" (UniqueName: \"kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h\") pod \"network-check-target-kcfzq\" (UID: \"49219a1b-1827-48a1-96c0-35b6b6695373\") " pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:35.266744 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:35.266708 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:13:35.266744 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:35.266744 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:13:35.266938 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:35.266759 2574 projected.go:196] Error preparing data for projected volume kube-api-access-crh8h for pod openshift-network-diagnostics/network-check-target-kcfzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:35.266938 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:35.266827 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h podName:49219a1b-1827-48a1-96c0-35b6b6695373 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:43.2668081 +0000 UTC m=+18.321229736 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-crh8h" (UniqueName: "kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h") pod "network-check-target-kcfzq" (UID: "49219a1b-1827-48a1-96c0-35b6b6695373") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:35.433519 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:35.433172 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:35.433519 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:35.433320 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:35.433519 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:35.433328 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:35.433519 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:35.433416 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:36.426608 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:36.426521 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:36.427107 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:36.426675 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:37.426102 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:37.425864 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:37.426102 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:37.425864 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:37.426102 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:37.426008 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:37.426102 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:37.426096 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:38.425989 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:38.425946 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:38.426468 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:38.426096 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:39.426324 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:39.426215 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:39.426324 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:39.426288 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:39.426791 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:39.426376 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:39.426791 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:39.426509 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:39.515699 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:39.515657 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" event={"ID":"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502","Type":"ContainerStarted","Data":"327b7765cb35cdbd6b095d78b7c22c436c7da19e5e47a9ae1a32d974a86541a8"} Jul 07 13:13:39.517447 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:39.517416 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zjrkx" event={"ID":"ff29c277-a852-463a-8e9f-e066329a2095","Type":"ContainerStarted","Data":"8bc7d94a128220342b543b64b8ac676b5122a1acd534a1776e466ce0b6927351"} Jul 07 13:13:39.519198 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:39.519163 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-txrsh" event={"ID":"3829f252-2935-4883-9ce3-b3823a2c3572","Type":"ContainerStarted","Data":"54bad6f2ddbc737e773ff18eb02f3ae4ad8fb340141c405f168516bfaf5d6bc9"} Jul 07 13:13:39.535748 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:39.535685 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zjrkx" podStartSLOduration=2.368015305 podStartE2EDuration="13.535664033s" totalImagesPullingTime="11.167648728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:39.535149012 +0000 UTC m=+14.589570697" watchObservedRunningTime="2026-07-07 13:13:39.535664033 +0000 UTC m=+14.590085693" Jul 07 13:13:39.550803 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:39.550746 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-txrsh" podStartSLOduration=2.309080049 podStartE2EDuration="13.550727958s" totalImagesPullingTime="11.241647909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:39.550675267 +0000 UTC m=+14.605096939" watchObservedRunningTime="2026-07-07 13:13:39.550727958 +0000 UTC m=+14.605149615" Jul 07 13:13:39.618785 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:39.618709 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:39.619437 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:39.619395 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:40.426148 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:40.426106 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:40.426370 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:40.426231 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:41.426347 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:41.426305 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:41.426549 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:41.426352 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:41.426549 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:41.426464 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:41.427009 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:41.426582 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:41.523419 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:41.523391 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 13:13:42.426070 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:42.426027 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:42.426330 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:42.426184 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:43.132507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:43.132461 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:43.133069 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:43.132593 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:43.133069 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:43.132668 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs podName:fefb5fd3-beb1-426e-b2e0-6defab925a6a nodeName:}" failed. No retries permitted until 2026-07-07 13:13:59.132645262 +0000 UTC m=+34.187066921 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs") pod "network-metrics-daemon-gcblz" (UID: "fefb5fd3-beb1-426e-b2e0-6defab925a6a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:43.233183 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:43.233142 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:43.233392 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:43.233346 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:43.233462 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:43.233432 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret podName:2bb5f676-7523-4cab-b7de-fd7cbd99581d nodeName:}" failed. No retries permitted until 2026-07-07 13:13:59.233410315 +0000 UTC m=+34.287831965 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret") pod "global-pull-secret-syncer-tq86p" (UID: "2bb5f676-7523-4cab-b7de-fd7cbd99581d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:43.333947 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:43.333911 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crh8h\" (UniqueName: \"kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h\") pod \"network-check-target-kcfzq\" (UID: \"49219a1b-1827-48a1-96c0-35b6b6695373\") " pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:43.334131 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:43.334093 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:13:43.334131 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:43.334118 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:13:43.334131 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:43.334128 2574 projected.go:196] Error preparing data for projected volume kube-api-access-crh8h for pod openshift-network-diagnostics/network-check-target-kcfzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:43.334352 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:43.334186 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h podName:49219a1b-1827-48a1-96c0-35b6b6695373 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:59.334171699 +0000 UTC m=+34.388593335 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-crh8h" (UniqueName: "kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h") pod "network-check-target-kcfzq" (UID: "49219a1b-1827-48a1-96c0-35b6b6695373") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:43.426809 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:43.426727 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:43.426809 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:43.426727 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:43.427045 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:43.426930 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:43.427161 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:43.427094 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:44.426479 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:44.426436 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:44.426918 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:44.426602 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:45.427539 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:45.427496 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:45.428020 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:45.427604 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:45.428020 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:45.427695 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:45.428020 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:45.427806 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:46.426084 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:46.426041 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:46.426301 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:46.426172 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:47.426616 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:47.426575 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:47.426616 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:47.426623 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:47.427102 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:47.426724 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:47.427159 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:47.427130 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:47.778892 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:47.778843 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 13:13:48.426566 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.426379 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:48.426741 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:48.426644 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:48.431474 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.431385 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T13:13:47.778871318Z","UUID":"2e70298d-5d4f-4ee4-a262-9a5f9a6ca113","Handler":null,"Name":"","Endpoint":""} Jul 07 13:13:48.432938 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.432920 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 13:13:48.433017 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.432956 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 13:13:48.539300 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.539205 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8tndt" event={"ID":"bdead414-6f59-4be1-8d50-131a7d61342d","Type":"ContainerStarted","Data":"8b053706184f3e1594882781ed706af7929ac97646076ab7ba6027cc0d4a92d3"} Jul 07 13:13:48.542170 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.542129 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" event={"ID":"d5eb2177-1423-49f0-b2b1-a583e0646586","Type":"ContainerStarted","Data":"09ecbe2a97872a19a08e74718160eab112f9dff782bfb9f1dc9fea655b3bca84"} Jul 07 13:13:48.542335 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.542202 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" event={"ID":"d5eb2177-1423-49f0-b2b1-a583e0646586","Type":"ContainerStarted","Data":"7476bb7cdb4f2b41e21bae23ee018f225fee0a5f142b604c825c980a633f3a99"} Jul 07 13:13:48.542335 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.542242 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" event={"ID":"d5eb2177-1423-49f0-b2b1-a583e0646586","Type":"ContainerStarted","Data":"37ef44a7b6bd90ccfb83fa01ab59da407d07682cd41ea714bcdbe7ad16c0ac9d"} Jul 07 13:13:48.542335 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.542296 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" event={"ID":"d5eb2177-1423-49f0-b2b1-a583e0646586","Type":"ContainerStarted","Data":"41e70028aff7a62d250a48fe6b7438815fd18c407dc9dc9f181b662d303ac4e0"} Jul 07 13:13:48.542335 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.542330 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" event={"ID":"d5eb2177-1423-49f0-b2b1-a583e0646586","Type":"ContainerStarted","Data":"67d06388983878ed1d313d674c11bd2bb7cc3e1bb608f1409227e99a1f5a59f9"} Jul 07 13:13:48.542520 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.542375 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" event={"ID":"d5eb2177-1423-49f0-b2b1-a583e0646586","Type":"ContainerStarted","Data":"92f038501d890d346871c6ff11c7d15b8d6680dcdb62a05371657139cb020f42"} Jul 07 13:13:48.543802 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.543774 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" event={"ID":"72ecc126-5631-4fa3-a4bd-4e636dd7e04f","Type":"ContainerStarted","Data":"b95b8d29cedcf6604bd10517f5146e18c899e4c5f603a043511fde42379d30cd"} Jul 07 13:13:48.545174 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.545151 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tzl89" event={"ID":"c9f3c64c-d89e-4583-955e-66d1c534336d","Type":"ContainerStarted","Data":"f7ead5bf45ac332a6fc38cb115ea049063c27a445a32ea1611fd448fd26978c6"} Jul 07 13:13:48.546708 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.546687 2574 generic.go:356] "Generic (PLEG): container finished" podID="78b4321d-698d-48e8-b2d7-be1e8f43ccee" containerID="e5560433208035b2a70be98d5ad1d03a475c33f4c3bcb3efac08278a79552966" exitCode=0 Jul 07 13:13:48.546829 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.546752 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwk42" event={"ID":"78b4321d-698d-48e8-b2d7-be1e8f43ccee","Type":"ContainerDied","Data":"e5560433208035b2a70be98d5ad1d03a475c33f4c3bcb3efac08278a79552966"} Jul 07 13:13:48.548547 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.548529 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" event={"ID":"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502","Type":"ContainerStarted","Data":"01c01c12fbc571d0d3d9a71b054d2203e8e95bfefaf4591d96840aa29ae16bb3"} Jul 07 13:13:48.559895 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.559849 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8tndt" podStartSLOduration=2.68597864 podStartE2EDuration="22.559836588s" totalImagesPullingTime="19.873857948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:48.559388897 +0000 UTC m=+23.613810571" watchObservedRunningTime="2026-07-07 13:13:48.559836588 +0000 UTC m=+23.614258245" Jul 07 13:13:48.577380 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.577319 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7h5hc" podStartSLOduration=2.6785528039999997 podStartE2EDuration="22.577297106s" totalImagesPullingTime="19.898744302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:48.577159499 +0000 UTC m=+23.631581182" watchObservedRunningTime="2026-07-07 13:13:48.577297106 +0000 UTC m=+23.631718769" Jul 07 13:13:48.623425 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:48.623367 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tzl89" podStartSLOduration=22.623340496 podStartE2EDuration="22.623340496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-07 13:13:48.594575365 +0000 UTC m=+23.648997025" watchObservedRunningTime="2026-07-07 13:13:48.623340496 +0000 UTC m=+23.677762158" Jul 07 13:13:49.426196 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:49.425934 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:49.426196 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:49.425944 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:49.426196 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:49.426067 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:49.426196 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:49.426127 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:49.444300 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:49.444268 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:49.444764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:49.444413 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 13:13:49.444999 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:49.444976 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zjrkx" Jul 07 13:13:49.551870 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:49.551830 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r6zsr" event={"ID":"062eca81-d98c-4875-b05a-783b32893f22","Type":"ContainerStarted","Data":"42afaee4f551fd6c523c955c37b354c47ff9158a33656c6c7aec3bd3f52ae189"} Jul 07 13:13:49.556859 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:49.556812 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" event={"ID":"34ba3f19-8b1e-4d3d-904c-ef07ce4f5502","Type":"ContainerStarted","Data":"ebcf42bef12f53247485752e47c765433a0e96761e647987aebd10aa691dabd2"} Jul 07 13:13:49.569906 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:49.569859 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r6zsr" podStartSLOduration=12.362699719 podStartE2EDuration="23.569843669s" totalImagesPullingTime="11.20714395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:49.569197454 +0000 UTC m=+24.623619113" watchObservedRunningTime="2026-07-07 13:13:49.569843669 +0000 UTC m=+24.624265358" Jul 07 13:13:50.426593 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:50.426557 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:50.426785 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:50.426672 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:50.564384 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:50.564297 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" event={"ID":"d5eb2177-1423-49f0-b2b1-a583e0646586","Type":"ContainerStarted","Data":"449e969f90c81ea07bcfd2d67faf92ec07e7cc9132769119325ccf438afac9cc"} Jul 07 13:13:51.426909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:51.426713 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:51.427219 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:51.426713 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:51.427219 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:51.427010 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:51.427219 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:51.427147 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:52.426495 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:52.426468 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:52.426867 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:52.426571 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:52.573921 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:52.573498 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" event={"ID":"d5eb2177-1423-49f0-b2b1-a583e0646586","Type":"ContainerStarted","Data":"0674dcb506d014a489776a1f5b5ac0c8d45b1065070e7315c8693455be93dc96"} Jul 07 13:13:52.573921 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:52.573869 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:52.592994 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:52.592602 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:52.604133 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:52.604070 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" podStartSLOduration=6.712401912 podStartE2EDuration="26.604048073s" totalImagesPullingTime="19.891646161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:52.603754232 +0000 UTC m=+27.658175916" watchObservedRunningTime="2026-07-07 13:13:52.604048073 +0000 UTC m=+27.658469731" Jul 07 13:13:52.604425 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:52.604392 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kkkzp" podStartSLOduration=5.39157322 podStartE2EDuration="26.604383335s" totalImagesPullingTime="21.212810115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:49.594441994 +0000 UTC m=+24.648863669" watchObservedRunningTime="2026-07-07 13:13:52.604383335 +0000 UTC m=+27.658805022" Jul 07 13:13:53.426430 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:53.426191 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:53.426609 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:53.426317 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:53.426609 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:53.426536 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:53.427389 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:53.426609 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:53.575945 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:53.575910 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:53.576111 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:53.576028 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:53.593481 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:53.593451 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:13:54.426466 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:54.426429 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:54.426685 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:54.426591 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:54.611451 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:54.611413 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tq86p"] Jul 07 13:13:54.611638 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:54.611544 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:54.611697 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:54.611660 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:54.614930 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:54.614895 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gcblz"] Jul 07 13:13:54.615082 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:54.615034 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:54.615168 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:54.615149 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:54.617890 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:54.617862 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kcfzq"] Jul 07 13:13:54.618007 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:54.617955 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:54.618051 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:54.618033 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:55.581999 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:55.581961 2574 generic.go:356] "Generic (PLEG): container finished" podID="78b4321d-698d-48e8-b2d7-be1e8f43ccee" containerID="35491b11e2739b11cfb00c802799ff502d13de6a850b949a1af84e2a7656c496" exitCode=0 Jul 07 13:13:55.582522 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:55.582055 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwk42" event={"ID":"78b4321d-698d-48e8-b2d7-be1e8f43ccee","Type":"ContainerDied","Data":"35491b11e2739b11cfb00c802799ff502d13de6a850b949a1af84e2a7656c496"} Jul 07 13:13:56.426288 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:56.426237 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:56.426471 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:56.426358 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:56.426471 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:56.426362 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:56.426602 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:56.426476 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:56.426602 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:56.426364 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:56.426602 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:56.426578 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:57.590142 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:57.589950 2574 generic.go:356] "Generic (PLEG): container finished" podID="78b4321d-698d-48e8-b2d7-be1e8f43ccee" containerID="7dfa08072ae3c4fd418d7b07d5c1aba817ac2f7c3a1a05157f1739ff437937fc" exitCode=0 Jul 07 13:13:57.590533 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:57.590034 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwk42" event={"ID":"78b4321d-698d-48e8-b2d7-be1e8f43ccee","Type":"ContainerDied","Data":"7dfa08072ae3c4fd418d7b07d5c1aba817ac2f7c3a1a05157f1739ff437937fc"} Jul 07 13:13:58.425894 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:58.425864 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:58.425989 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:58.425968 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:58.425989 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:58.425974 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcfzq" podUID="49219a1b-1827-48a1-96c0-35b6b6695373" Jul 07 13:13:58.426066 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:58.425999 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:58.426099 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:58.426079 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tq86p" podUID="2bb5f676-7523-4cab-b7de-fd7cbd99581d" Jul 07 13:13:58.426168 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:58.426153 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gcblz" podUID="fefb5fd3-beb1-426e-b2e0-6defab925a6a" Jul 07 13:13:58.595100 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:58.595014 2574 generic.go:356] "Generic (PLEG): container finished" podID="78b4321d-698d-48e8-b2d7-be1e8f43ccee" containerID="01514393702a12747515a01c8980a4cca980f02cda6fc7fb747024496bf8b29b" exitCode=0 Jul 07 13:13:58.595100 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:58.595076 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwk42" event={"ID":"78b4321d-698d-48e8-b2d7-be1e8f43ccee","Type":"ContainerDied","Data":"01514393702a12747515a01c8980a4cca980f02cda6fc7fb747024496bf8b29b"} Jul 07 13:13:59.145687 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:59.145641 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:13:59.145854 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:59.145801 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:59.145916 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:59.145866 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs podName:fefb5fd3-beb1-426e-b2e0-6defab925a6a nodeName:}" failed. No retries permitted until 2026-07-07 13:14:31.145850003 +0000 UTC m=+66.200271639 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs") pod "network-metrics-daemon-gcblz" (UID: "fefb5fd3-beb1-426e-b2e0-6defab925a6a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:13:59.246058 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:59.246014 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:13:59.246217 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:59.246141 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:59.246217 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:59.246193 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret podName:2bb5f676-7523-4cab-b7de-fd7cbd99581d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:31.246180062 +0000 UTC m=+66.300601699 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret") pod "global-pull-secret-syncer-tq86p" (UID: "2bb5f676-7523-4cab-b7de-fd7cbd99581d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:13:59.346554 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:13:59.346517 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crh8h\" (UniqueName: \"kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h\") pod \"network-check-target-kcfzq\" (UID: \"49219a1b-1827-48a1-96c0-35b6b6695373\") " pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:13:59.346728 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:59.346662 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:13:59.346728 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:59.346679 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:13:59.346728 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:59.346690 2574 projected.go:196] Error preparing data for projected volume kube-api-access-crh8h for pod openshift-network-diagnostics/network-check-target-kcfzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:13:59.346850 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:13:59.346755 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h podName:49219a1b-1827-48a1-96c0-35b6b6695373 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:31.346737367 +0000 UTC m=+66.401159005 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-crh8h" (UniqueName: "kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h") pod "network-check-target-kcfzq" (UID: "49219a1b-1827-48a1-96c0-35b6b6695373") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:14:00.270865 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.270829 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-95.ec2.internal" event="NodeReady" Jul 07 13:14:00.271416 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.271004 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 13:14:00.343896 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.343810 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-87b889985-54znm"] Jul 07 13:14:00.345497 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.345481 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.349907 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.349881 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 13:14:00.350024 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.349969 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 13:14:00.350812 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.350791 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 13:14:00.359306 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.359284 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s5lng\"" Jul 07 13:14:00.359955 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.359939 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 13:14:00.394183 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.394152 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-87b889985-54znm"] Jul 07 13:14:00.395199 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.395165 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r6464"] Jul 07 13:14:00.397057 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.397032 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-d4zmg"] Jul 07 13:14:00.397213 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.397185 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.398912 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.398888 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d4zmg" Jul 07 13:14:00.426166 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.426136 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:14:00.426377 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.426136 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:14:00.426449 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.426145 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:14:00.433344 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.433319 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 13:14:00.442500 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.442472 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 13:14:00.442640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.442560 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qwjzx\"" Jul 07 13:14:00.442757 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.442740 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5ztml\"" Jul 07 13:14:00.442757 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.442756 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 13:14:00.443975 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.443955 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 13:14:00.444506 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.444400 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 13:14:00.444506 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.444394 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fjddr\"" Jul 07 13:14:00.444506 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.444471 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 13:14:00.444506 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.444474 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 13:14:00.444896 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.444878 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mbgxn\"" Jul 07 13:14:00.445147 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.445131 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 13:14:00.456161 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.456121 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d4zmg"] Jul 07 13:14:00.456360 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.456314 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-certificates\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.456432 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.456392 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.456485 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.456428 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-image-registry-private-configuration\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.456485 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.456459 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-trusted-ca\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.456584 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.456545 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8beaeed0-113a-49fc-84b9-8a1e16193878-ca-trust-extracted\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.456629 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.456592 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-installation-pull-secrets\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.456673 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.456626 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-bound-sa-token\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.456673 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.456656 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5nm6\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-kube-api-access-g5nm6\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.465695 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.465665 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 13:14:00.480483 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.480446 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r6464"] Jul 07 13:14:00.557789 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.557748 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.557789 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.557787 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-image-registry-private-configuration\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.558041 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:00.557890 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:14:00.558041 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:00.557905 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b889985-54znm: secret "image-registry-tls" not found Jul 07 13:14:00.558041 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.557908 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-config-volume\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.558041 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.557940 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.558041 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:00.557973 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls podName:8beaeed0-113a-49fc-84b9-8a1e16193878 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:01.057951249 +0000 UTC m=+36.112372888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls") pod "image-registry-87b889985-54znm" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878") : secret "image-registry-tls" not found Jul 07 13:14:00.558305 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.558039 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-trusted-ca\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.558305 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.558095 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8beaeed0-113a-49fc-84b9-8a1e16193878-ca-trust-extracted\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.558305 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.558126 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-installation-pull-secrets\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.558305 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.558146 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-bound-sa-token\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.558305 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.558303 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5nm6\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-kube-api-access-g5nm6\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.558500 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.558329 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-tmp-dir\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.558500 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.558357 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf-cert\") pod \"ingress-canary-d4zmg\" (UID: \"64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf\") " pod="openshift-ingress-canary/ingress-canary-d4zmg" Jul 07 13:14:00.558500 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.558388 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-224hx\" (UniqueName: \"kubernetes.io/projected/64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf-kube-api-access-224hx\") pod \"ingress-canary-d4zmg\" (UID: \"64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf\") " pod="openshift-ingress-canary/ingress-canary-d4zmg" Jul 07 13:14:00.558500 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.558415 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gskn\" (UniqueName: \"kubernetes.io/projected/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-kube-api-access-8gskn\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.558500 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.558449 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-certificates\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.558700 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.558558 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8beaeed0-113a-49fc-84b9-8a1e16193878-ca-trust-extracted\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.559075 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.559043 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-certificates\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.559328 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.559305 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-trusted-ca\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.562589 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.562567 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-installation-pull-secrets\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.562665 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.562567 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-image-registry-private-configuration\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.568372 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.568345 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-bound-sa-token\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.568507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.568345 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5nm6\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-kube-api-access-g5nm6\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:00.659693 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.659608 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-config-volume\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.659693 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.659644 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.659693 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.659686 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-tmp-dir\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.659926 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.659701 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf-cert\") pod \"ingress-canary-d4zmg\" (UID: \"64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf\") " pod="openshift-ingress-canary/ingress-canary-d4zmg" Jul 07 13:14:00.659926 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.659719 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-224hx\" (UniqueName: \"kubernetes.io/projected/64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf-kube-api-access-224hx\") pod \"ingress-canary-d4zmg\" (UID: \"64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf\") " pod="openshift-ingress-canary/ingress-canary-d4zmg" Jul 07 13:14:00.659926 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.659734 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gskn\" (UniqueName: \"kubernetes.io/projected/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-kube-api-access-8gskn\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.659926 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:00.659849 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:14:00.660107 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:00.659962 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls podName:aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:01.159943204 +0000 UTC m=+36.214364852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls") pod "dns-default-r6464" (UID: "aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d") : secret "dns-default-metrics-tls" not found Jul 07 13:14:00.660107 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.660080 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-tmp-dir\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.660354 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.660333 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-config-volume\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.662258 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.662229 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf-cert\") pod \"ingress-canary-d4zmg\" (UID: \"64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf\") " pod="openshift-ingress-canary/ingress-canary-d4zmg" Jul 07 13:14:00.673852 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.673820 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-224hx\" (UniqueName: \"kubernetes.io/projected/64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf-kube-api-access-224hx\") pod \"ingress-canary-d4zmg\" (UID: \"64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf\") " pod="openshift-ingress-canary/ingress-canary-d4zmg" Jul 07 13:14:00.677076 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.677045 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gskn\" (UniqueName: \"kubernetes.io/projected/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-kube-api-access-8gskn\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:00.716383 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.716337 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d4zmg" Jul 07 13:14:00.893136 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:00.892913 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d4zmg"] Jul 07 13:14:00.898661 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:00.898622 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64b4c5e4_1a86_445f_9ae2_8fa8a1667ccf.slice/crio-5a5de51862f2289f1db576964d81bdcf5243ff7defac326775e142fb25358492 WatchSource:0}: Error finding container 5a5de51862f2289f1db576964d81bdcf5243ff7defac326775e142fb25358492: Status 404 returned error can't find the container with id 5a5de51862f2289f1db576964d81bdcf5243ff7defac326775e142fb25358492 Jul 07 13:14:01.062816 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:01.062770 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:01.063031 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:01.062982 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:14:01.063031 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:01.063007 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b889985-54znm: secret "image-registry-tls" not found Jul 07 13:14:01.063129 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:01.063088 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls podName:8beaeed0-113a-49fc-84b9-8a1e16193878 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:02.063065097 +0000 UTC m=+37.117486756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls") pod "image-registry-87b889985-54znm" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878") : secret "image-registry-tls" not found Jul 07 13:14:01.163542 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:01.163501 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:01.163716 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:01.163638 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:14:01.163716 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:01.163705 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls podName:aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:02.163688409 +0000 UTC m=+37.218110059 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls") pod "dns-default-r6464" (UID: "aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d") : secret "dns-default-metrics-tls" not found Jul 07 13:14:01.605127 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:01.605089 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d4zmg" event={"ID":"64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf","Type":"ContainerStarted","Data":"5a5de51862f2289f1db576964d81bdcf5243ff7defac326775e142fb25358492"} Jul 07 13:14:02.074057 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:02.074014 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:02.074419 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:02.074194 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:14:02.074419 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:02.074218 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b889985-54znm: secret "image-registry-tls" not found Jul 07 13:14:02.074419 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:02.074304 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls podName:8beaeed0-113a-49fc-84b9-8a1e16193878 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:04.074281325 +0000 UTC m=+39.128702985 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls") pod "image-registry-87b889985-54znm" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878") : secret "image-registry-tls" not found Jul 07 13:14:02.175221 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:02.175179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:02.175454 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:02.175393 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:14:02.175520 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:02.175458 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls podName:aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:04.175437672 +0000 UTC m=+39.229859315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls") pod "dns-default-r6464" (UID: "aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d") : secret "dns-default-metrics-tls" not found Jul 07 13:14:04.092342 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:04.092297 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:04.092802 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:04.092487 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:14:04.092802 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:04.092514 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b889985-54znm: secret "image-registry-tls" not found Jul 07 13:14:04.092802 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:04.092594 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls podName:8beaeed0-113a-49fc-84b9-8a1e16193878 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:08.092568748 +0000 UTC m=+43.146990400 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls") pod "image-registry-87b889985-54znm" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878") : secret "image-registry-tls" not found Jul 07 13:14:04.192764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:04.192726 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:04.192954 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:04.192890 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:14:04.193012 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:04.192961 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls podName:aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:08.192942652 +0000 UTC m=+43.247364306 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls") pod "dns-default-r6464" (UID: "aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d") : secret "dns-default-metrics-tls" not found Jul 07 13:14:05.618189 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:05.618147 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwk42" event={"ID":"78b4321d-698d-48e8-b2d7-be1e8f43ccee","Type":"ContainerStarted","Data":"e029116bd106b535d711a11842f38b10929bcc2610cf2643c087958bef28de7d"} Jul 07 13:14:05.620160 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:05.620133 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d4zmg" event={"ID":"64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf","Type":"ContainerStarted","Data":"422dc9455b74d11c4c55b02deced0e25db9f1a00573d3ace8ad02868f67bb415"} Jul 07 13:14:05.662038 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:05.661986 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-d4zmg" podStartSLOduration=1.215448758 podStartE2EDuration="5.661969017s" totalImagesPullingTime="4.446520259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:05.661427135 +0000 UTC m=+40.715848814" watchObservedRunningTime="2026-07-07 13:14:05.661969017 +0000 UTC m=+40.716390676" Jul 07 13:14:06.625763 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:06.625725 2574 generic.go:356] "Generic (PLEG): container finished" podID="78b4321d-698d-48e8-b2d7-be1e8f43ccee" containerID="e029116bd106b535d711a11842f38b10929bcc2610cf2643c087958bef28de7d" exitCode=0 Jul 07 13:14:06.626150 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:06.625796 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwk42" event={"ID":"78b4321d-698d-48e8-b2d7-be1e8f43ccee","Type":"ContainerDied","Data":"e029116bd106b535d711a11842f38b10929bcc2610cf2643c087958bef28de7d"} Jul 07 13:14:07.630863 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:07.630825 2574 generic.go:356] "Generic (PLEG): container finished" podID="78b4321d-698d-48e8-b2d7-be1e8f43ccee" containerID="bbf46323f162fb65618fb1061c8eef39b7b56c9797baf424f8e52d515fb2d9cc" exitCode=0 Jul 07 13:14:07.631257 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:07.630886 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwk42" event={"ID":"78b4321d-698d-48e8-b2d7-be1e8f43ccee","Type":"ContainerDied","Data":"bbf46323f162fb65618fb1061c8eef39b7b56c9797baf424f8e52d515fb2d9cc"} Jul 07 13:14:08.124430 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:08.124384 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:08.124622 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:08.124543 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:14:08.124622 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:08.124565 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b889985-54znm: secret "image-registry-tls" not found Jul 07 13:14:08.124691 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:08.124627 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls podName:8beaeed0-113a-49fc-84b9-8a1e16193878 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:16.124609201 +0000 UTC m=+51.179030842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls") pod "image-registry-87b889985-54znm" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878") : secret "image-registry-tls" not found Jul 07 13:14:08.224869 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:08.224826 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:08.224999 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:08.224975 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:14:08.225052 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:08.225040 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls podName:aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:16.225023464 +0000 UTC m=+51.279445101 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls") pod "dns-default-r6464" (UID: "aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d") : secret "dns-default-metrics-tls" not found Jul 07 13:14:08.636318 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:08.636285 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwk42" event={"ID":"78b4321d-698d-48e8-b2d7-be1e8f43ccee","Type":"ContainerStarted","Data":"254c23d03d90137af44f4ac3bcd18f729fe013ebbd8663330cd737a1ad32697b"} Jul 07 13:14:08.665896 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:08.665832 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hwk42" podStartSLOduration=16.514997789 podStartE2EDuration="42.6658147s" totalImagesPullingTime="26.150816911s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:08.664031813 +0000 UTC m=+43.718453490" watchObservedRunningTime="2026-07-07 13:14:08.6658147 +0000 UTC m=+43.720236357" Jul 07 13:14:15.291926 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.291887 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-58547f567f-h6ff4"] Jul 07 13:14:15.332175 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.332126 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58547f567f-h6ff4"] Jul 07 13:14:15.332378 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.332278 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.336542 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.336513 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 13:14:15.336697 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.336588 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 13:14:15.336862 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.336846 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 13:14:15.336924 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.336909 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-cdt7x\"" Jul 07 13:14:15.337394 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.337377 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 13:14:15.337496 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.337417 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 13:14:15.337496 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.337451 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 13:14:15.376855 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.376797 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-stats-auth\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.377058 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.376903 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.377058 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.376948 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-default-certificate\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.377058 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.377030 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.377182 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.377065 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z278n\" (UniqueName: \"kubernetes.io/projected/6d6726a9-eb0f-45db-b9e9-5068cbda8413-kube-api-access-z278n\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.477562 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.477517 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.477562 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.477570 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-default-certificate\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.477805 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:15.477724 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:15.977705731 +0000 UTC m=+51.032127367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : configmap references non-existent config key: service-ca.crt Jul 07 13:14:15.477805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.477757 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.477805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.477787 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z278n\" (UniqueName: \"kubernetes.io/projected/6d6726a9-eb0f-45db-b9e9-5068cbda8413-kube-api-access-z278n\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.477964 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.477829 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-stats-auth\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.477964 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:15.477923 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 13:14:15.478062 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:15.478002 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:15.97798374 +0000 UTC m=+51.032405395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : secret "router-metrics-certs-default" not found Jul 07 13:14:15.481449 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.481423 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-stats-auth\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.481592 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.481423 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-default-certificate\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.487645 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.487613 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z278n\" (UniqueName: \"kubernetes.io/projected/6d6726a9-eb0f-45db-b9e9-5068cbda8413-kube-api-access-z278n\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.983228 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.983191 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.983469 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:15.983282 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:15.983469 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:15.983381 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 13:14:15.983469 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:15.983413 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:16.983388639 +0000 UTC m=+52.037810282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : configmap references non-existent config key: service-ca.crt Jul 07 13:14:15.983469 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:15.983446 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:16.98343593 +0000 UTC m=+52.037857573 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : secret "router-metrics-certs-default" not found Jul 07 13:14:16.184272 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:16.184213 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:16.184466 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:16.184378 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:14:16.184466 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:16.184402 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b889985-54znm: secret "image-registry-tls" not found Jul 07 13:14:16.184466 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:16.184461 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls podName:8beaeed0-113a-49fc-84b9-8a1e16193878 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:32.184442927 +0000 UTC m=+67.238864563 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls") pod "image-registry-87b889985-54znm" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878") : secret "image-registry-tls" not found Jul 07 13:14:16.284745 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:16.284649 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:16.284896 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:16.284800 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:14:16.284896 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:16.284872 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls podName:aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d nodeName:}" failed. No retries permitted until 2026-07-07 13:14:32.284854707 +0000 UTC m=+67.339276343 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls") pod "dns-default-r6464" (UID: "aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d") : secret "dns-default-metrics-tls" not found Jul 07 13:14:16.990642 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:16.990606 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:16.991110 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:16.990701 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:16.991110 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:16.990767 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 13:14:16.991110 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:16.990831 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:18.990815177 +0000 UTC m=+54.045236813 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : secret "router-metrics-certs-default" not found Jul 07 13:14:16.991110 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:16.990849 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:18.990840974 +0000 UTC m=+54.045262609 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : configmap references non-existent config key: service-ca.crt Jul 07 13:14:19.005478 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:19.005432 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:19.006142 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:19.005534 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:19.006142 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:19.005611 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 13:14:19.006142 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:19.005669 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:23.005630902 +0000 UTC m=+58.060052553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : configmap references non-existent config key: service-ca.crt Jul 07 13:14:19.006142 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:19.005740 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:23.005719924 +0000 UTC m=+58.060141561 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : secret "router-metrics-certs-default" not found Jul 07 13:14:20.277670 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.277631 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r"] Jul 07 13:14:20.279870 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.279849 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" Jul 07 13:14:20.282299 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.282271 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 13:14:20.282443 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.282278 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:14:20.282748 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.282729 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-zp7n4\"" Jul 07 13:14:20.283264 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.283232 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 13:14:20.283337 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.283322 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 13:14:20.295532 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.295497 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r"] Jul 07 13:14:20.318831 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.318794 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f569c81-186b-46b5-a6a3-7d10df78af9f-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-g4q8r\" (UID: \"7f569c81-186b-46b5-a6a3-7d10df78af9f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" Jul 07 13:14:20.319026 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.318886 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f569c81-186b-46b5-a6a3-7d10df78af9f-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-g4q8r\" (UID: \"7f569c81-186b-46b5-a6a3-7d10df78af9f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" Jul 07 13:14:20.319026 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.318916 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsr5n\" (UniqueName: \"kubernetes.io/projected/7f569c81-186b-46b5-a6a3-7d10df78af9f-kube-api-access-fsr5n\") pod \"kube-storage-version-migrator-operator-846bf668f7-g4q8r\" (UID: \"7f569c81-186b-46b5-a6a3-7d10df78af9f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" Jul 07 13:14:20.420000 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.419959 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f569c81-186b-46b5-a6a3-7d10df78af9f-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-g4q8r\" (UID: \"7f569c81-186b-46b5-a6a3-7d10df78af9f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" Jul 07 13:14:20.420000 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.419998 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsr5n\" (UniqueName: \"kubernetes.io/projected/7f569c81-186b-46b5-a6a3-7d10df78af9f-kube-api-access-fsr5n\") pod \"kube-storage-version-migrator-operator-846bf668f7-g4q8r\" (UID: \"7f569c81-186b-46b5-a6a3-7d10df78af9f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" Jul 07 13:14:20.420226 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.420203 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f569c81-186b-46b5-a6a3-7d10df78af9f-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-g4q8r\" (UID: \"7f569c81-186b-46b5-a6a3-7d10df78af9f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" Jul 07 13:14:20.420750 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.420725 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f569c81-186b-46b5-a6a3-7d10df78af9f-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-g4q8r\" (UID: \"7f569c81-186b-46b5-a6a3-7d10df78af9f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" Jul 07 13:14:20.422351 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.422329 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f569c81-186b-46b5-a6a3-7d10df78af9f-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-g4q8r\" (UID: \"7f569c81-186b-46b5-a6a3-7d10df78af9f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" Jul 07 13:14:20.429307 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.429277 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsr5n\" (UniqueName: \"kubernetes.io/projected/7f569c81-186b-46b5-a6a3-7d10df78af9f-kube-api-access-fsr5n\") pod \"kube-storage-version-migrator-operator-846bf668f7-g4q8r\" (UID: \"7f569c81-186b-46b5-a6a3-7d10df78af9f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" Jul 07 13:14:20.589147 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.589046 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" Jul 07 13:14:20.719160 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.719123 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r"] Jul 07 13:14:20.977805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.977759 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mccwm"] Jul 07 13:14:20.980119 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.980099 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:20.982607 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.982581 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 13:14:20.983463 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.983439 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 13:14:20.983463 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.983448 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wkhkp\"" Jul 07 13:14:20.983463 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.983439 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 13:14:20.983693 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.983482 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 13:14:20.997551 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:20.997518 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mccwm"] Jul 07 13:14:21.025587 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.025539 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e144ab3c-4492-409c-9ac1-06fad984629b-crio-socket\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.025587 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.025582 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.025835 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.025638 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e144ab3c-4492-409c-9ac1-06fad984629b-data-volume\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.025835 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.025668 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rqg2\" (UniqueName: \"kubernetes.io/projected/e144ab3c-4492-409c-9ac1-06fad984629b-kube-api-access-9rqg2\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.025835 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.025721 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.126334 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.126277 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e144ab3c-4492-409c-9ac1-06fad984629b-data-volume\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.126334 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.126338 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rqg2\" (UniqueName: \"kubernetes.io/projected/e144ab3c-4492-409c-9ac1-06fad984629b-kube-api-access-9rqg2\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.126614 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.126379 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.126614 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.126436 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e144ab3c-4492-409c-9ac1-06fad984629b-crio-socket\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.126614 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.126459 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.126614 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:21.126510 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 13:14:21.126614 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.126531 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e144ab3c-4492-409c-9ac1-06fad984629b-crio-socket\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.126614 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:21.126574 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 13:14:21.126614 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:21.126607 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm podName:e144ab3c-4492-409c-9ac1-06fad984629b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:21.626582525 +0000 UTC m=+56.681004182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mccwm" (UID: "e144ab3c-4492-409c-9ac1-06fad984629b") : configmap "kube-rbac-proxy" not found Jul 07 13:14:21.126921 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:21.126626 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls podName:e144ab3c-4492-409c-9ac1-06fad984629b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:21.62661745 +0000 UTC m=+56.681039086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mccwm" (UID: "e144ab3c-4492-409c-9ac1-06fad984629b") : secret "insights-runtime-extractor-tls" not found Jul 07 13:14:21.126921 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.126627 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e144ab3c-4492-409c-9ac1-06fad984629b-data-volume\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.135832 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.135802 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rqg2\" (UniqueName: \"kubernetes.io/projected/e144ab3c-4492-409c-9ac1-06fad984629b-kube-api-access-9rqg2\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.630829 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.630786 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.631340 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.630886 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:21.631340 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:21.630966 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 13:14:21.631340 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:21.631008 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 13:14:21.631340 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:21.631080 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm podName:e144ab3c-4492-409c-9ac1-06fad984629b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:22.63106046 +0000 UTC m=+57.685482115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mccwm" (UID: "e144ab3c-4492-409c-9ac1-06fad984629b") : configmap "kube-rbac-proxy" not found Jul 07 13:14:21.631340 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:21.631094 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls podName:e144ab3c-4492-409c-9ac1-06fad984629b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:22.63108847 +0000 UTC m=+57.685510105 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mccwm" (UID: "e144ab3c-4492-409c-9ac1-06fad984629b") : secret "insights-runtime-extractor-tls" not found Jul 07 13:14:21.668128 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:21.668092 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" event={"ID":"7f569c81-186b-46b5-a6a3-7d10df78af9f","Type":"ContainerStarted","Data":"4eab46f8bd37d54ab95de4fb5edca75529d2ce0ac8d65dd49a81fbb47f97f36f"} Jul 07 13:14:22.262017 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.261976 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn"] Jul 07 13:14:22.265872 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.264491 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" Jul 07 13:14:22.267381 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.267357 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 13:14:22.267457 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.267396 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-n9zvc\"" Jul 07 13:14:22.268134 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.268115 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 13:14:22.268202 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.268188 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:14:22.268242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.268216 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 13:14:22.279613 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.279585 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn"] Jul 07 13:14:22.336863 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.336822 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1137478a-31b6-40e6-a2d3-c7222e4680f1-config\") pod \"service-ca-operator-5d46d9ffd4-69fmn\" (UID: \"1137478a-31b6-40e6-a2d3-c7222e4680f1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" Jul 07 13:14:22.336863 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.336870 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj8n6\" (UniqueName: \"kubernetes.io/projected/1137478a-31b6-40e6-a2d3-c7222e4680f1-kube-api-access-bj8n6\") pod \"service-ca-operator-5d46d9ffd4-69fmn\" (UID: \"1137478a-31b6-40e6-a2d3-c7222e4680f1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" Jul 07 13:14:22.337157 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.337001 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1137478a-31b6-40e6-a2d3-c7222e4680f1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-69fmn\" (UID: \"1137478a-31b6-40e6-a2d3-c7222e4680f1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" Jul 07 13:14:22.438374 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.438340 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1137478a-31b6-40e6-a2d3-c7222e4680f1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-69fmn\" (UID: \"1137478a-31b6-40e6-a2d3-c7222e4680f1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" Jul 07 13:14:22.438566 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.438476 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1137478a-31b6-40e6-a2d3-c7222e4680f1-config\") pod \"service-ca-operator-5d46d9ffd4-69fmn\" (UID: \"1137478a-31b6-40e6-a2d3-c7222e4680f1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" Jul 07 13:14:22.438566 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.438506 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj8n6\" (UniqueName: \"kubernetes.io/projected/1137478a-31b6-40e6-a2d3-c7222e4680f1-kube-api-access-bj8n6\") pod \"service-ca-operator-5d46d9ffd4-69fmn\" (UID: \"1137478a-31b6-40e6-a2d3-c7222e4680f1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" Jul 07 13:14:22.439014 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.438989 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1137478a-31b6-40e6-a2d3-c7222e4680f1-config\") pod \"service-ca-operator-5d46d9ffd4-69fmn\" (UID: \"1137478a-31b6-40e6-a2d3-c7222e4680f1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" Jul 07 13:14:22.440929 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.440906 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1137478a-31b6-40e6-a2d3-c7222e4680f1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-69fmn\" (UID: \"1137478a-31b6-40e6-a2d3-c7222e4680f1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" Jul 07 13:14:22.449157 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.449126 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj8n6\" (UniqueName: \"kubernetes.io/projected/1137478a-31b6-40e6-a2d3-c7222e4680f1-kube-api-access-bj8n6\") pod \"service-ca-operator-5d46d9ffd4-69fmn\" (UID: \"1137478a-31b6-40e6-a2d3-c7222e4680f1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" Jul 07 13:14:22.575858 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.575782 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" Jul 07 13:14:22.640699 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.640654 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:22.641146 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.640811 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:22.641146 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:22.640834 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 13:14:22.641146 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:22.640928 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls podName:e144ab3c-4492-409c-9ac1-06fad984629b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:24.640905628 +0000 UTC m=+59.695327281 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mccwm" (UID: "e144ab3c-4492-409c-9ac1-06fad984629b") : secret "insights-runtime-extractor-tls" not found Jul 07 13:14:22.641146 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:22.640945 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 13:14:22.641146 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:22.641015 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm podName:e144ab3c-4492-409c-9ac1-06fad984629b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:24.640997846 +0000 UTC m=+59.695419493 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mccwm" (UID: "e144ab3c-4492-409c-9ac1-06fad984629b") : configmap "kube-rbac-proxy" not found Jul 07 13:14:22.705507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.705465 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn"] Jul 07 13:14:22.708701 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:22.708668 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1137478a_31b6_40e6_a2d3_c7222e4680f1.slice/crio-79615b2ccd04a96c27fdb0805443c3a960fdcea2c360b1a62fcfc62ccfc7c640 WatchSource:0}: Error finding container 79615b2ccd04a96c27fdb0805443c3a960fdcea2c360b1a62fcfc62ccfc7c640: Status 404 returned error can't find the container with id 79615b2ccd04a96c27fdb0805443c3a960fdcea2c360b1a62fcfc62ccfc7c640 Jul 07 13:14:22.774058 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:22.774027 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tzl89_c9f3c64c-d89e-4583-955e-66d1c534336d/dns-node-resolver/0.log" Jul 07 13:14:23.044418 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:23.044381 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:23.044634 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:23.044484 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:23.044634 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:23.044552 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 13:14:23.044634 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:23.044633 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:31.044616094 +0000 UTC m=+66.099037734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : configmap references non-existent config key: service-ca.crt Jul 07 13:14:23.044805 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:23.044653 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:31.044646848 +0000 UTC m=+66.099068483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : secret "router-metrics-certs-default" not found Jul 07 13:14:23.374823 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:23.374738 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-txrsh_3829f252-2935-4883-9ce3-b3823a2c3572/node-ca/0.log" Jul 07 13:14:23.675634 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:23.675533 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" event={"ID":"7f569c81-186b-46b5-a6a3-7d10df78af9f","Type":"ContainerStarted","Data":"118f15bc6e4553e15c3b8b59adc90b91b9fe30df343cfd3070f15ce49b6e3559"} Jul 07 13:14:23.676775 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:23.676748 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" event={"ID":"1137478a-31b6-40e6-a2d3-c7222e4680f1","Type":"ContainerStarted","Data":"79615b2ccd04a96c27fdb0805443c3a960fdcea2c360b1a62fcfc62ccfc7c640"} Jul 07 13:14:23.694906 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:23.694845 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" podStartSLOduration=0.914831303 podStartE2EDuration="3.694826647s" totalImagesPullingTime="2.779995344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:23.693972788 +0000 UTC m=+58.748394449" watchObservedRunningTime="2026-07-07 13:14:23.694826647 +0000 UTC m=+58.749248304" Jul 07 13:14:24.173975 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:24.173909 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d4zmg_64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf/serve-healthcheck-canary/0.log" Jul 07 13:14:24.661975 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:24.661932 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:24.662176 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:24.662010 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:24.662176 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:24.662070 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 13:14:24.662176 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:24.662134 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 13:14:24.662309 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:24.662144 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm podName:e144ab3c-4492-409c-9ac1-06fad984629b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:28.662126547 +0000 UTC m=+63.716548184 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mccwm" (UID: "e144ab3c-4492-409c-9ac1-06fad984629b") : configmap "kube-rbac-proxy" not found Jul 07 13:14:24.662309 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:24.662206 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls podName:e144ab3c-4492-409c-9ac1-06fad984629b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:28.662189872 +0000 UTC m=+63.716611512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mccwm" (UID: "e144ab3c-4492-409c-9ac1-06fad984629b") : secret "insights-runtime-extractor-tls" not found Jul 07 13:14:25.599236 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:25.599202 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h48b6" Jul 07 13:14:26.686076 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:26.686036 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" event={"ID":"1137478a-31b6-40e6-a2d3-c7222e4680f1","Type":"ContainerStarted","Data":"1557be1e3e46e98d50624e1165fef3303aed4a651bbef30dbe191a94d04af26c"} Jul 07 13:14:26.703161 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:26.703096 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" podStartSLOduration=1.6852678779999999 podStartE2EDuration="4.703074808s" totalImagesPullingTime="3.01780693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:26.70263882 +0000 UTC m=+61.757060478" watchObservedRunningTime="2026-07-07 13:14:26.703074808 +0000 UTC m=+61.757496469" Jul 07 13:14:28.698162 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:28.698107 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:28.698656 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:28.698204 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:28.698656 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:28.698235 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 13:14:28.698656 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:28.698321 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm podName:e144ab3c-4492-409c-9ac1-06fad984629b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:36.698304792 +0000 UTC m=+71.752726428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mccwm" (UID: "e144ab3c-4492-409c-9ac1-06fad984629b") : configmap "kube-rbac-proxy" not found Jul 07 13:14:28.698656 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:28.698330 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 13:14:28.698656 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:28.698379 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls podName:e144ab3c-4492-409c-9ac1-06fad984629b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:36.698364183 +0000 UTC m=+71.752785818 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mccwm" (UID: "e144ab3c-4492-409c-9ac1-06fad984629b") : secret "insights-runtime-extractor-tls" not found Jul 07 13:14:30.056258 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.056209 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-89m5v"] Jul 07 13:14:30.058963 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.058942 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-89m5v" Jul 07 13:14:30.061448 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.061423 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qjh8l\"" Jul 07 13:14:30.061569 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.061423 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 07 13:14:30.062336 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.062297 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 07 13:14:30.062336 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.062306 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 07 13:14:30.062507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.062353 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 07 13:14:30.067326 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.067302 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-89m5v"] Jul 07 13:14:30.110560 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.110518 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/608e5d27-47f7-4f55-ab10-c209332d6c4f-signing-key\") pod \"service-ca-758f599c5-89m5v\" (UID: \"608e5d27-47f7-4f55-ab10-c209332d6c4f\") " pod="openshift-service-ca/service-ca-758f599c5-89m5v" Jul 07 13:14:30.110560 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.110555 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/608e5d27-47f7-4f55-ab10-c209332d6c4f-signing-cabundle\") pod \"service-ca-758f599c5-89m5v\" (UID: \"608e5d27-47f7-4f55-ab10-c209332d6c4f\") " pod="openshift-service-ca/service-ca-758f599c5-89m5v" Jul 07 13:14:30.110839 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.110580 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4cb6\" (UniqueName: \"kubernetes.io/projected/608e5d27-47f7-4f55-ab10-c209332d6c4f-kube-api-access-t4cb6\") pod \"service-ca-758f599c5-89m5v\" (UID: \"608e5d27-47f7-4f55-ab10-c209332d6c4f\") " pod="openshift-service-ca/service-ca-758f599c5-89m5v" Jul 07 13:14:30.211423 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.211381 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/608e5d27-47f7-4f55-ab10-c209332d6c4f-signing-key\") pod \"service-ca-758f599c5-89m5v\" (UID: \"608e5d27-47f7-4f55-ab10-c209332d6c4f\") " pod="openshift-service-ca/service-ca-758f599c5-89m5v" Jul 07 13:14:30.211423 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.211420 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/608e5d27-47f7-4f55-ab10-c209332d6c4f-signing-cabundle\") pod \"service-ca-758f599c5-89m5v\" (UID: \"608e5d27-47f7-4f55-ab10-c209332d6c4f\") " pod="openshift-service-ca/service-ca-758f599c5-89m5v" Jul 07 13:14:30.211628 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.211452 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4cb6\" (UniqueName: \"kubernetes.io/projected/608e5d27-47f7-4f55-ab10-c209332d6c4f-kube-api-access-t4cb6\") pod \"service-ca-758f599c5-89m5v\" (UID: \"608e5d27-47f7-4f55-ab10-c209332d6c4f\") " pod="openshift-service-ca/service-ca-758f599c5-89m5v" Jul 07 13:14:30.212051 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.212028 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/608e5d27-47f7-4f55-ab10-c209332d6c4f-signing-cabundle\") pod \"service-ca-758f599c5-89m5v\" (UID: \"608e5d27-47f7-4f55-ab10-c209332d6c4f\") " pod="openshift-service-ca/service-ca-758f599c5-89m5v" Jul 07 13:14:30.213943 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.213919 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/608e5d27-47f7-4f55-ab10-c209332d6c4f-signing-key\") pod \"service-ca-758f599c5-89m5v\" (UID: \"608e5d27-47f7-4f55-ab10-c209332d6c4f\") " pod="openshift-service-ca/service-ca-758f599c5-89m5v" Jul 07 13:14:30.220499 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.220473 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4cb6\" (UniqueName: \"kubernetes.io/projected/608e5d27-47f7-4f55-ab10-c209332d6c4f-kube-api-access-t4cb6\") pod \"service-ca-758f599c5-89m5v\" (UID: \"608e5d27-47f7-4f55-ab10-c209332d6c4f\") " pod="openshift-service-ca/service-ca-758f599c5-89m5v" Jul 07 13:14:30.368586 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.368479 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-89m5v" Jul 07 13:14:30.498301 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.498260 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-89m5v"] Jul 07 13:14:30.501725 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:30.501684 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod608e5d27_47f7_4f55_ab10_c209332d6c4f.slice/crio-52ccaefe64804d15e488f74882aa0bef897aab358b8b84fd6888ae42a75c4496 WatchSource:0}: Error finding container 52ccaefe64804d15e488f74882aa0bef897aab358b8b84fd6888ae42a75c4496: Status 404 returned error can't find the container with id 52ccaefe64804d15e488f74882aa0bef897aab358b8b84fd6888ae42a75c4496 Jul 07 13:14:30.697779 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.697690 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-89m5v" event={"ID":"608e5d27-47f7-4f55-ab10-c209332d6c4f","Type":"ContainerStarted","Data":"bbf0077ac394e3b657149cd5ad3e7eace818fc9dec7c4c2f07bde72ed8b84f0e"} Jul 07 13:14:30.697779 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.697744 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-89m5v" event={"ID":"608e5d27-47f7-4f55-ab10-c209332d6c4f","Type":"ContainerStarted","Data":"52ccaefe64804d15e488f74882aa0bef897aab358b8b84fd6888ae42a75c4496"} Jul 07 13:14:30.718370 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:30.718316 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-89m5v" podStartSLOduration=0.718297757 podStartE2EDuration="718.297757ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:30.717626291 +0000 UTC m=+65.772047949" watchObservedRunningTime="2026-07-07 13:14:30.718297757 +0000 UTC m=+65.772719414" Jul 07 13:14:31.119613 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.119567 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:31.120133 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:31.119746 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 13:14:31.120133 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.119771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:31.120133 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:31.119829 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:47.11980603 +0000 UTC m=+82.174227669 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : secret "router-metrics-certs-default" not found Jul 07 13:14:31.120133 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:31.119894 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle podName:6d6726a9-eb0f-45db-b9e9-5068cbda8413 nodeName:}" failed. No retries permitted until 2026-07-07 13:14:47.119878415 +0000 UTC m=+82.174300058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle") pod "router-default-58547f567f-h6ff4" (UID: "6d6726a9-eb0f-45db-b9e9-5068cbda8413") : configmap references non-existent config key: service-ca.crt Jul 07 13:14:31.220633 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.220600 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:14:31.225176 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.225151 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 13:14:31.231534 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:31.231508 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 13:14:31.231646 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:31.231583 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs podName:fefb5fd3-beb1-426e-b2e0-6defab925a6a nodeName:}" failed. No retries permitted until 2026-07-07 13:15:35.231564069 +0000 UTC m=+130.285985707 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs") pod "network-metrics-daemon-gcblz" (UID: "fefb5fd3-beb1-426e-b2e0-6defab925a6a") : secret "metrics-daemon-secret" not found Jul 07 13:14:31.321974 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.321922 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:14:31.324718 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.324697 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 13:14:31.334566 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.334538 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb5f676-7523-4cab-b7de-fd7cbd99581d-original-pull-secret\") pod \"global-pull-secret-syncer-tq86p\" (UID: \"2bb5f676-7523-4cab-b7de-fd7cbd99581d\") " pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:14:31.351985 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.351948 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tq86p" Jul 07 13:14:31.422981 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.422949 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crh8h\" (UniqueName: \"kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h\") pod \"network-check-target-kcfzq\" (UID: \"49219a1b-1827-48a1-96c0-35b6b6695373\") " pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:14:31.425891 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.425862 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 13:14:31.437670 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.437470 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 13:14:31.446706 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.446676 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crh8h\" (UniqueName: \"kubernetes.io/projected/49219a1b-1827-48a1-96c0-35b6b6695373-kube-api-access-crh8h\") pod \"network-check-target-kcfzq\" (UID: \"49219a1b-1827-48a1-96c0-35b6b6695373\") " pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:14:31.620629 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.620593 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tq86p"] Jul 07 13:14:31.623822 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:31.623788 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb5f676_7523_4cab_b7de_fd7cbd99581d.slice/crio-4ddbea714d8c771d1e851e1bced021cb3c71b29cbbd9898c89d7b38b33017d0c WatchSource:0}: Error finding container 4ddbea714d8c771d1e851e1bced021cb3c71b29cbbd9898c89d7b38b33017d0c: Status 404 returned error can't find the container with id 4ddbea714d8c771d1e851e1bced021cb3c71b29cbbd9898c89d7b38b33017d0c Jul 07 13:14:31.641031 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.641003 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5ztml\"" Jul 07 13:14:31.648442 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.648406 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:14:31.702335 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.702242 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tq86p" event={"ID":"2bb5f676-7523-4cab-b7de-fd7cbd99581d","Type":"ContainerStarted","Data":"4ddbea714d8c771d1e851e1bced021cb3c71b29cbbd9898c89d7b38b33017d0c"} Jul 07 13:14:31.808937 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:31.808899 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kcfzq"] Jul 07 13:14:31.813051 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:31.813019 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49219a1b_1827_48a1_96c0_35b6b6695373.slice/crio-d7f8bc4c09c24d6fef5ad1426fac3af48e3744c3524b3a924b5451ed266b9c69 WatchSource:0}: Error finding container d7f8bc4c09c24d6fef5ad1426fac3af48e3744c3524b3a924b5451ed266b9c69: Status 404 returned error can't find the container with id d7f8bc4c09c24d6fef5ad1426fac3af48e3744c3524b3a924b5451ed266b9c69 Jul 07 13:14:32.230672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:32.230630 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:14:32.231164 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:32.230817 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:14:32.231164 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:32.230842 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b889985-54znm: secret "image-registry-tls" not found Jul 07 13:14:32.231164 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:32.230913 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls podName:8beaeed0-113a-49fc-84b9-8a1e16193878 nodeName:}" failed. No retries permitted until 2026-07-07 13:15:04.230891763 +0000 UTC m=+99.285313405 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls") pod "image-registry-87b889985-54znm" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878") : secret "image-registry-tls" not found Jul 07 13:14:32.332264 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:32.331867 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:14:32.332264 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:32.332026 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:14:32.332264 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:32.332090 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls podName:aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d nodeName:}" failed. No retries permitted until 2026-07-07 13:15:04.33207326 +0000 UTC m=+99.386494921 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls") pod "dns-default-r6464" (UID: "aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d") : secret "dns-default-metrics-tls" not found Jul 07 13:14:32.707067 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:32.707025 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kcfzq" event={"ID":"49219a1b-1827-48a1-96c0-35b6b6695373","Type":"ContainerStarted","Data":"d7f8bc4c09c24d6fef5ad1426fac3af48e3744c3524b3a924b5451ed266b9c69"} Jul 07 13:14:35.718152 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:35.718057 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kcfzq" event={"ID":"49219a1b-1827-48a1-96c0-35b6b6695373","Type":"ContainerStarted","Data":"29672f49befe890e24d426353555817b84ec0fefc7f2824f55e4c53cfffdf02a"} Jul 07 13:14:35.718653 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:35.718154 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:14:35.737955 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:35.737890 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kcfzq" podStartSLOduration=66.090799068 podStartE2EDuration="1m9.737868444s" totalImagesPullingTime="3.647069376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:35.737444476 +0000 UTC m=+70.791866195" watchObservedRunningTime="2026-07-07 13:14:35.737868444 +0000 UTC m=+70.792290104" Jul 07 13:14:36.776324 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:36.776284 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:36.776810 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:36.776367 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:36.776810 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:36.776467 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 13:14:36.776810 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:14:36.776522 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls podName:e144ab3c-4492-409c-9ac1-06fad984629b nodeName:}" failed. No retries permitted until 2026-07-07 13:14:52.776508534 +0000 UTC m=+87.830930170 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mccwm" (UID: "e144ab3c-4492-409c-9ac1-06fad984629b") : secret "insights-runtime-extractor-tls" not found Jul 07 13:14:36.776942 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:36.776812 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e144ab3c-4492-409c-9ac1-06fad984629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:39.731562 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:39.731511 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tq86p" event={"ID":"2bb5f676-7523-4cab-b7de-fd7cbd99581d","Type":"ContainerStarted","Data":"285a26482cee9ad3f1448e5de8314852bcd8d8dcbb381e52bce232f58639a5f2"} Jul 07 13:14:39.750452 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:39.750391 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tq86p" podStartSLOduration=66.65087145 podStartE2EDuration="1m13.750373946s" totalImagesPullingTime="7.099502496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:39.749712418 +0000 UTC m=+74.804134098" watchObservedRunningTime="2026-07-07 13:14:39.750373946 +0000 UTC m=+74.804795603" Jul 07 13:14:47.169170 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:47.169131 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:47.169696 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:47.169216 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:47.170416 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:47.170397 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d6726a9-eb0f-45db-b9e9-5068cbda8413-service-ca-bundle\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:47.171590 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:47.171569 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d6726a9-eb0f-45db-b9e9-5068cbda8413-metrics-certs\") pod \"router-default-58547f567f-h6ff4\" (UID: \"6d6726a9-eb0f-45db-b9e9-5068cbda8413\") " pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:47.444623 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:47.444474 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-cdt7x\"" Jul 07 13:14:47.452332 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:47.452290 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:47.585176 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:47.585138 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58547f567f-h6ff4"] Jul 07 13:14:47.588679 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:47.588650 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d6726a9_eb0f_45db_b9e9_5068cbda8413.slice/crio-27752a7e4f6515c6702ea1333bfe332875208029df73923be9247a31ee1d158e WatchSource:0}: Error finding container 27752a7e4f6515c6702ea1333bfe332875208029df73923be9247a31ee1d158e: Status 404 returned error can't find the container with id 27752a7e4f6515c6702ea1333bfe332875208029df73923be9247a31ee1d158e Jul 07 13:14:47.757848 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:47.757810 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58547f567f-h6ff4" event={"ID":"6d6726a9-eb0f-45db-b9e9-5068cbda8413","Type":"ContainerStarted","Data":"313dccab7fd9d2df24fba94974ee5043099353c60aa2797ec311f82f175eb935"} Jul 07 13:14:47.758077 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:47.757896 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58547f567f-h6ff4" event={"ID":"6d6726a9-eb0f-45db-b9e9-5068cbda8413","Type":"ContainerStarted","Data":"27752a7e4f6515c6702ea1333bfe332875208029df73923be9247a31ee1d158e"} Jul 07 13:14:47.778662 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:47.778508 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-58547f567f-h6ff4" podStartSLOduration=32.778487887 podStartE2EDuration="32.778487887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:47.778455414 +0000 UTC m=+82.832877104" watchObservedRunningTime="2026-07-07 13:14:47.778487887 +0000 UTC m=+82.832909548" Jul 07 13:14:48.453119 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:48.453078 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:48.455784 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:48.455759 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:48.761179 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:48.761142 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:48.762481 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:48.762459 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-58547f567f-h6ff4" Jul 07 13:14:52.809618 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:52.809565 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:52.811945 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:52.811919 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e144ab3c-4492-409c-9ac1-06fad984629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mccwm\" (UID: \"e144ab3c-4492-409c-9ac1-06fad984629b\") " pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:53.051522 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.051459 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm"] Jul 07 13:14:53.054185 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.054158 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:14:53.057924 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.057888 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 13:14:53.057924 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.057921 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 13:14:53.058127 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.057930 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 13:14:53.058127 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.057940 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 13:14:53.061391 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.061313 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t"] Jul 07 13:14:53.063327 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.063307 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t" Jul 07 13:14:53.065809 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.065784 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 07 13:14:53.065936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.065817 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-j4c7s\"" Jul 07 13:14:53.072195 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.072167 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm"] Jul 07 13:14:53.077834 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.077800 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t"] Jul 07 13:14:53.093585 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.093552 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wkhkp\"" Jul 07 13:14:53.101406 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.101372 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mccwm" Jul 07 13:14:53.112683 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.112645 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmvl2\" (UniqueName: \"kubernetes.io/projected/bd5ef562-4442-4499-9e7c-8c4ff005b9db-kube-api-access-xmvl2\") pod \"klusterlet-addon-workmgr-7569d86f7d-w4bpm\" (UID: \"bd5ef562-4442-4499-9e7c-8c4ff005b9db\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:14:53.112805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.112697 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bd5ef562-4442-4499-9e7c-8c4ff005b9db-klusterlet-config\") pod \"klusterlet-addon-workmgr-7569d86f7d-w4bpm\" (UID: \"bd5ef562-4442-4499-9e7c-8c4ff005b9db\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:14:53.112805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.112741 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f5nv\" (UniqueName: \"kubernetes.io/projected/6af0e983-6e15-4f7c-919c-ad3c2fcf1110-kube-api-access-2f5nv\") pod \"managed-serviceaccount-addon-agent-5b9c5f664b-rp55t\" (UID: \"6af0e983-6e15-4f7c-919c-ad3c2fcf1110\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t" Jul 07 13:14:53.112805 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.112792 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6af0e983-6e15-4f7c-919c-ad3c2fcf1110-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b9c5f664b-rp55t\" (UID: \"6af0e983-6e15-4f7c-919c-ad3c2fcf1110\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t" Jul 07 13:14:53.112959 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.112817 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd5ef562-4442-4499-9e7c-8c4ff005b9db-tmp\") pod \"klusterlet-addon-workmgr-7569d86f7d-w4bpm\" (UID: \"bd5ef562-4442-4499-9e7c-8c4ff005b9db\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:14:53.173826 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.173787 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx"] Jul 07 13:14:53.178559 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.177997 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx" Jul 07 13:14:53.183045 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.183017 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 13:14:53.183045 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.183039 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vwhbf\"" Jul 07 13:14:53.193788 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.193760 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7978659b89-5cb4w"] Jul 07 13:14:53.195748 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.195724 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-2jh4q"] Jul 07 13:14:53.195901 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.195832 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.198459 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.198013 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-2jh4q" Jul 07 13:14:53.201033 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.200879 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 13:14:53.201900 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.201874 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx"] Jul 07 13:14:53.207654 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.207386 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 13:14:53.210788 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.210765 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 13:14:53.211559 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.211178 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 13:14:53.211559 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.211185 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 13:14:53.211559 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.211377 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9qjtb\"" Jul 07 13:14:53.211559 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.211449 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dkwl9\"" Jul 07 13:14:53.211559 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.211493 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 13:14:53.211559 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.211549 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 13:14:53.213236 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213214 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-oauth-serving-cert\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.213375 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213274 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9acd5758-f64c-4851-8475-5b684fdc8d55-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-chbmx\" (UID: \"9acd5758-f64c-4851-8475-5b684fdc8d55\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx" Jul 07 13:14:53.213375 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213323 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-oauth-config\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.213375 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213346 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-serving-cert\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.213491 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213395 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-service-ca\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.213524 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213497 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmvl2\" (UniqueName: \"kubernetes.io/projected/bd5ef562-4442-4499-9e7c-8c4ff005b9db-kube-api-access-xmvl2\") pod \"klusterlet-addon-workmgr-7569d86f7d-w4bpm\" (UID: \"bd5ef562-4442-4499-9e7c-8c4ff005b9db\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:14:53.213559 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213534 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bd5ef562-4442-4499-9e7c-8c4ff005b9db-klusterlet-config\") pod \"klusterlet-addon-workmgr-7569d86f7d-w4bpm\" (UID: \"bd5ef562-4442-4499-9e7c-8c4ff005b9db\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:14:53.213592 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213578 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f5nv\" (UniqueName: \"kubernetes.io/projected/6af0e983-6e15-4f7c-919c-ad3c2fcf1110-kube-api-access-2f5nv\") pod \"managed-serviceaccount-addon-agent-5b9c5f664b-rp55t\" (UID: \"6af0e983-6e15-4f7c-919c-ad3c2fcf1110\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t" Jul 07 13:14:53.213625 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213603 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dv6j\" (UniqueName: \"kubernetes.io/projected/006691fe-1d4c-41fd-82a8-740043a25930-kube-api-access-8dv6j\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.213674 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213630 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-console-config\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.213674 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213663 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljbs5\" (UniqueName: \"kubernetes.io/projected/4efdac5c-6234-4954-9fe4-5b2d5dcb2303-kube-api-access-ljbs5\") pod \"downloads-685bfb5f64-2jh4q\" (UID: \"4efdac5c-6234-4954-9fe4-5b2d5dcb2303\") " pod="openshift-console/downloads-685bfb5f64-2jh4q" Jul 07 13:14:53.213787 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213770 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6af0e983-6e15-4f7c-919c-ad3c2fcf1110-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b9c5f664b-rp55t\" (UID: \"6af0e983-6e15-4f7c-919c-ad3c2fcf1110\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t" Jul 07 13:14:53.213843 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.213801 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd5ef562-4442-4499-9e7c-8c4ff005b9db-tmp\") pod \"klusterlet-addon-workmgr-7569d86f7d-w4bpm\" (UID: \"bd5ef562-4442-4499-9e7c-8c4ff005b9db\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:14:53.214302 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.214276 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd5ef562-4442-4499-9e7c-8c4ff005b9db-tmp\") pod \"klusterlet-addon-workmgr-7569d86f7d-w4bpm\" (UID: \"bd5ef562-4442-4499-9e7c-8c4ff005b9db\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:14:53.216084 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.216053 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bd5ef562-4442-4499-9e7c-8c4ff005b9db-klusterlet-config\") pod \"klusterlet-addon-workmgr-7569d86f7d-w4bpm\" (UID: \"bd5ef562-4442-4499-9e7c-8c4ff005b9db\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:14:53.216321 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.216273 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6af0e983-6e15-4f7c-919c-ad3c2fcf1110-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b9c5f664b-rp55t\" (UID: \"6af0e983-6e15-4f7c-919c-ad3c2fcf1110\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t" Jul 07 13:14:53.228185 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.228146 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-2jh4q"] Jul 07 13:14:53.229281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.229235 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7978659b89-5cb4w"] Jul 07 13:14:53.242295 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.242259 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f5nv\" (UniqueName: \"kubernetes.io/projected/6af0e983-6e15-4f7c-919c-ad3c2fcf1110-kube-api-access-2f5nv\") pod \"managed-serviceaccount-addon-agent-5b9c5f664b-rp55t\" (UID: \"6af0e983-6e15-4f7c-919c-ad3c2fcf1110\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t" Jul 07 13:14:53.244276 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.244234 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmvl2\" (UniqueName: \"kubernetes.io/projected/bd5ef562-4442-4499-9e7c-8c4ff005b9db-kube-api-access-xmvl2\") pod \"klusterlet-addon-workmgr-7569d86f7d-w4bpm\" (UID: \"bd5ef562-4442-4499-9e7c-8c4ff005b9db\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:14:53.287380 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.287354 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mccwm"] Jul 07 13:14:53.288875 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:53.288848 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode144ab3c_4492_409c_9ac1_06fad984629b.slice/crio-a5c84aedb06a30f5db980db76343195161d8d31c73b6b841a5a0ccde6c589c52 WatchSource:0}: Error finding container a5c84aedb06a30f5db980db76343195161d8d31c73b6b841a5a0ccde6c589c52: Status 404 returned error can't find the container with id a5c84aedb06a30f5db980db76343195161d8d31c73b6b841a5a0ccde6c589c52 Jul 07 13:14:53.314131 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.314030 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dv6j\" (UniqueName: \"kubernetes.io/projected/006691fe-1d4c-41fd-82a8-740043a25930-kube-api-access-8dv6j\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.314131 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.314062 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-console-config\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.314131 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.314089 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljbs5\" (UniqueName: \"kubernetes.io/projected/4efdac5c-6234-4954-9fe4-5b2d5dcb2303-kube-api-access-ljbs5\") pod \"downloads-685bfb5f64-2jh4q\" (UID: \"4efdac5c-6234-4954-9fe4-5b2d5dcb2303\") " pod="openshift-console/downloads-685bfb5f64-2jh4q" Jul 07 13:14:53.314131 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.314119 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-oauth-serving-cert\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.314486 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.314152 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9acd5758-f64c-4851-8475-5b684fdc8d55-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-chbmx\" (UID: \"9acd5758-f64c-4851-8475-5b684fdc8d55\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx" Jul 07 13:14:53.314486 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.314211 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-oauth-config\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.314486 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.314272 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-serving-cert\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.314486 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.314301 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-service-ca\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.315429 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.315404 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-console-config\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.315556 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.315411 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-service-ca\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.315627 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.315576 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-oauth-serving-cert\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.317013 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.316985 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-oauth-config\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.317129 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.317047 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-serving-cert\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.317129 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.317110 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9acd5758-f64c-4851-8475-5b684fdc8d55-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-chbmx\" (UID: \"9acd5758-f64c-4851-8475-5b684fdc8d55\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx" Jul 07 13:14:53.331351 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.331309 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dv6j\" (UniqueName: \"kubernetes.io/projected/006691fe-1d4c-41fd-82a8-740043a25930-kube-api-access-8dv6j\") pod \"console-7978659b89-5cb4w\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.332074 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.332054 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljbs5\" (UniqueName: \"kubernetes.io/projected/4efdac5c-6234-4954-9fe4-5b2d5dcb2303-kube-api-access-ljbs5\") pod \"downloads-685bfb5f64-2jh4q\" (UID: \"4efdac5c-6234-4954-9fe4-5b2d5dcb2303\") " pod="openshift-console/downloads-685bfb5f64-2jh4q" Jul 07 13:14:53.365048 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.365010 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:14:53.379956 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.379927 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t" Jul 07 13:14:53.490372 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.490067 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx" Jul 07 13:14:53.509432 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.509396 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:14:53.515675 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.515642 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm"] Jul 07 13:14:53.515675 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.515663 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-2jh4q" Jul 07 13:14:53.518739 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:53.518704 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd5ef562_4442_4499_9e7c_8c4ff005b9db.slice/crio-8713ace489d29f3423e84e6be6f6778fbfdb6e5c30059e5287ccad5151b44bc3 WatchSource:0}: Error finding container 8713ace489d29f3423e84e6be6f6778fbfdb6e5c30059e5287ccad5151b44bc3: Status 404 returned error can't find the container with id 8713ace489d29f3423e84e6be6f6778fbfdb6e5c30059e5287ccad5151b44bc3 Jul 07 13:14:53.528686 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.528566 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t"] Jul 07 13:14:53.530951 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:53.530913 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6af0e983_6e15_4f7c_919c_ad3c2fcf1110.slice/crio-e76a35bb6a2dc350e204161aeb71c6d51b985ffb1c988542e3bcccde8e87a061 WatchSource:0}: Error finding container e76a35bb6a2dc350e204161aeb71c6d51b985ffb1c988542e3bcccde8e87a061: Status 404 returned error can't find the container with id e76a35bb6a2dc350e204161aeb71c6d51b985ffb1c988542e3bcccde8e87a061 Jul 07 13:14:53.723039 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.722933 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx"] Jul 07 13:14:53.730345 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:53.730305 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9acd5758_f64c_4851_8475_5b684fdc8d55.slice/crio-a77c0b2b629ce18a2ff8aafad588c4a32b125a1be6de1ee017f930ab5e7b2535 WatchSource:0}: Error finding container a77c0b2b629ce18a2ff8aafad588c4a32b125a1be6de1ee017f930ab5e7b2535: Status 404 returned error can't find the container with id a77c0b2b629ce18a2ff8aafad588c4a32b125a1be6de1ee017f930ab5e7b2535 Jul 07 13:14:53.764411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.764377 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-2jh4q"] Jul 07 13:14:53.766229 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:53.766195 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4efdac5c_6234_4954_9fe4_5b2d5dcb2303.slice/crio-6e58e445ff837ee97e0ed16d4aa7a9c9d16de763032faec73c57b1ff00ad62a6 WatchSource:0}: Error finding container 6e58e445ff837ee97e0ed16d4aa7a9c9d16de763032faec73c57b1ff00ad62a6: Status 404 returned error can't find the container with id 6e58e445ff837ee97e0ed16d4aa7a9c9d16de763032faec73c57b1ff00ad62a6 Jul 07 13:14:53.775623 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.775437 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7978659b89-5cb4w"] Jul 07 13:14:53.778115 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:14:53.778080 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod006691fe_1d4c_41fd_82a8_740043a25930.slice/crio-092d6d5c91db321fa93416aff1adb6be2fdec8412c35749625ac0baf8edd8955 WatchSource:0}: Error finding container 092d6d5c91db321fa93416aff1adb6be2fdec8412c35749625ac0baf8edd8955: Status 404 returned error can't find the container with id 092d6d5c91db321fa93416aff1adb6be2fdec8412c35749625ac0baf8edd8955 Jul 07 13:14:53.782538 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.782502 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" event={"ID":"bd5ef562-4442-4499-9e7c-8c4ff005b9db","Type":"ContainerStarted","Data":"8713ace489d29f3423e84e6be6f6778fbfdb6e5c30059e5287ccad5151b44bc3"} Jul 07 13:14:53.784895 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.784807 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mccwm" event={"ID":"e144ab3c-4492-409c-9ac1-06fad984629b","Type":"ContainerStarted","Data":"a5c84aedb06a30f5db980db76343195161d8d31c73b6b841a5a0ccde6c589c52"} Jul 07 13:14:53.786785 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.786751 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t" event={"ID":"6af0e983-6e15-4f7c-919c-ad3c2fcf1110","Type":"ContainerStarted","Data":"e76a35bb6a2dc350e204161aeb71c6d51b985ffb1c988542e3bcccde8e87a061"} Jul 07 13:14:53.788571 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.788538 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-2jh4q" event={"ID":"4efdac5c-6234-4954-9fe4-5b2d5dcb2303","Type":"ContainerStarted","Data":"6e58e445ff837ee97e0ed16d4aa7a9c9d16de763032faec73c57b1ff00ad62a6"} Jul 07 13:14:53.790179 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:53.790148 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx" event={"ID":"9acd5758-f64c-4851-8475-5b684fdc8d55","Type":"ContainerStarted","Data":"a77c0b2b629ce18a2ff8aafad588c4a32b125a1be6de1ee017f930ab5e7b2535"} Jul 07 13:14:54.807834 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:14:54.807793 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7978659b89-5cb4w" event={"ID":"006691fe-1d4c-41fd-82a8-740043a25930","Type":"ContainerStarted","Data":"092d6d5c91db321fa93416aff1adb6be2fdec8412c35749625ac0baf8edd8955"} Jul 07 13:15:02.848535 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:02.848451 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" event={"ID":"bd5ef562-4442-4499-9e7c-8c4ff005b9db","Type":"ContainerStarted","Data":"c5f781882e5d69c82f1dc7ae769aa44868215687a83363efcaf8524f0647cbc7"} Jul 07 13:15:02.849357 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:02.849316 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:15:02.851693 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:02.851237 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" podUID="bd5ef562-4442-4499-9e7c-8c4ff005b9db" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.14:8000/readyz\": dial tcp 10.132.0.14:8000: connect: connection refused" Jul 07 13:15:02.860798 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:02.860762 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t" event={"ID":"6af0e983-6e15-4f7c-919c-ad3c2fcf1110","Type":"ContainerStarted","Data":"e72130afd5e8cc537265f3786ac6fcce25e5952656ba09050fb8bb6f5b75f3e3"} Jul 07 13:15:02.867257 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:02.867158 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx" Jul 07 13:15:02.868363 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:02.868206 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" podStartSLOduration=0.687237464 podStartE2EDuration="9.868186462s" totalImagesPullingTime="9.180948998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:02.867760002 +0000 UTC m=+97.922181687" watchObservedRunningTime="2026-07-07 13:15:02.868186462 +0000 UTC m=+97.922608121" Jul 07 13:15:02.869723 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:02.869695 2574 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-d559d66d5-chbmx container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.132.0.16:8443/healthz\": dial tcp 10.132.0.16:8443: connect: connection refused" start-of-body= Jul 07 13:15:02.869861 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:02.869749 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx" podUID="9acd5758-f64c-4851-8475-5b684fdc8d55" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.132.0.16:8443/healthz\": dial tcp 10.132.0.16:8443: connect: connection refused" Jul 07 13:15:02.889623 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:02.889011 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b9c5f664b-rp55t" podStartSLOduration=0.717941102 podStartE2EDuration="9.888984398s" totalImagesPullingTime="9.171043296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:02.887783127 +0000 UTC m=+97.942204786" watchObservedRunningTime="2026-07-07 13:15:02.888984398 +0000 UTC m=+97.943406058" Jul 07 13:15:02.913074 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:02.912997 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx" podStartSLOduration=1.345747426 podStartE2EDuration="9.912973352s" totalImagesPullingTime="8.567225926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:02.911395978 +0000 UTC m=+97.965817664" watchObservedRunningTime="2026-07-07 13:15:02.912973352 +0000 UTC m=+97.967395006" Jul 07 13:15:03.873010 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:03.872964 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx" event={"ID":"9acd5758-f64c-4851-8475-5b684fdc8d55","Type":"ContainerStarted","Data":"54f1aa7907613d4e5b36dd74417f039c5e13bed95d0c4532a5def2ecf2f09084"} Jul 07 13:15:03.875196 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:03.875159 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mccwm" event={"ID":"e144ab3c-4492-409c-9ac1-06fad984629b","Type":"ContainerStarted","Data":"419d436de677a8eafbbf56183a2b6f6486c1e43806d6480492407a9a77ff2c2e"} Jul 07 13:15:03.875358 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:03.875232 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mccwm" event={"ID":"e144ab3c-4492-409c-9ac1-06fad984629b","Type":"ContainerStarted","Data":"5adbbab7adee29974fc08aa0ca90d07d18da1e0810c227b7ff61d0b87df5616e"} Jul 07 13:15:03.877236 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:03.877205 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7978659b89-5cb4w" event={"ID":"006691fe-1d4c-41fd-82a8-740043a25930","Type":"ContainerStarted","Data":"87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da"} Jul 07 13:15:03.878075 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:03.878052 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-chbmx" Jul 07 13:15:03.878201 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:03.878104 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7569d86f7d-w4bpm" Jul 07 13:15:03.896727 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:03.896661 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7978659b89-5cb4w" podStartSLOduration=1.971980874 podStartE2EDuration="10.896642005s" totalImagesPullingTime="8.924661131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:03.894804426 +0000 UTC m=+98.949226109" watchObservedRunningTime="2026-07-07 13:15:03.896642005 +0000 UTC m=+98.951063663" Jul 07 13:15:04.317353 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.317302 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:15:04.319963 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.319919 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") pod \"image-registry-87b889985-54znm\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:15:04.419238 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.418709 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:15:04.421951 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.421920 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d-metrics-tls\") pod \"dns-default-r6464\" (UID: \"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d\") " pod="openshift-dns/dns-default-r6464" Jul 07 13:15:04.560471 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.560430 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s5lng\"" Jul 07 13:15:04.568289 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.568184 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:15:04.612973 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.612698 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qwjzx\"" Jul 07 13:15:04.620235 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.620199 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r6464" Jul 07 13:15:04.769853 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.769768 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-87b889985-54znm"] Jul 07 13:15:04.774676 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:15:04.774634 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8beaeed0_113a_49fc_84b9_8a1e16193878.slice/crio-7d4d06f88b7ca8091044fafbd9e879879a7a2dcf9cf787c0475fab8230dadbbb WatchSource:0}: Error finding container 7d4d06f88b7ca8091044fafbd9e879879a7a2dcf9cf787c0475fab8230dadbbb: Status 404 returned error can't find the container with id 7d4d06f88b7ca8091044fafbd9e879879a7a2dcf9cf787c0475fab8230dadbbb Jul 07 13:15:04.817989 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.817898 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r6464"] Jul 07 13:15:04.829730 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:15:04.829690 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae54f2e_3f86_4f77_b5d3_e8c5c875fb1d.slice/crio-3fab8a3fe19bdcecfe9425b1bc773e2fb7085fc3e5c103ccf8ee04f02075d081 WatchSource:0}: Error finding container 3fab8a3fe19bdcecfe9425b1bc773e2fb7085fc3e5c103ccf8ee04f02075d081: Status 404 returned error can't find the container with id 3fab8a3fe19bdcecfe9425b1bc773e2fb7085fc3e5c103ccf8ee04f02075d081 Jul 07 13:15:04.884126 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.884085 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r6464" event={"ID":"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d","Type":"ContainerStarted","Data":"3fab8a3fe19bdcecfe9425b1bc773e2fb7085fc3e5c103ccf8ee04f02075d081"} Jul 07 13:15:04.886934 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.886896 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87b889985-54znm" event={"ID":"8beaeed0-113a-49fc-84b9-8a1e16193878","Type":"ContainerStarted","Data":"03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19"} Jul 07 13:15:04.887090 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.886952 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87b889985-54znm" event={"ID":"8beaeed0-113a-49fc-84b9-8a1e16193878","Type":"ContainerStarted","Data":"7d4d06f88b7ca8091044fafbd9e879879a7a2dcf9cf787c0475fab8230dadbbb"} Jul 07 13:15:04.910072 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:04.910002 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-87b889985-54znm" podStartSLOduration=98.909975375 podStartE2EDuration="1m38.909975375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:04.909288597 +0000 UTC m=+99.963710275" watchObservedRunningTime="2026-07-07 13:15:04.909975375 +0000 UTC m=+99.964397034" Jul 07 13:15:05.898449 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:05.898353 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:15:06.723584 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:06.723544 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kcfzq" Jul 07 13:15:10.284681 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:10.284648 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7978659b89-5cb4w"] Jul 07 13:15:10.959041 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:10.958924 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mz4st"] Jul 07 13:15:10.986681 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:10.986641 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:10.989575 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:10.989384 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 13:15:10.990312 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:10.990277 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 13:15:10.991080 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:10.991050 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 13:15:10.992297 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:10.991050 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 13:15:10.992297 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:10.991506 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-h2gvw\"" Jul 07 13:15:10.992297 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:10.991544 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 13:15:10.992297 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:10.991763 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 13:15:11.080410 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.080371 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlw29\" (UniqueName: \"kubernetes.io/projected/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-kube-api-access-nlw29\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.080620 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.080419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-sys\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.080620 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.080445 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-root\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.080620 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.080499 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.080620 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.080528 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-tls\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.080620 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.080554 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-textfile\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.080620 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.080607 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-wtmp\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.080916 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.080630 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.080916 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.080662 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-metrics-client-ca\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.182281 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-wtmp\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.182339 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.182372 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-metrics-client-ca\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.182428 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlw29\" (UniqueName: \"kubernetes.io/projected/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-kube-api-access-nlw29\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.182460 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-sys\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.182468 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-wtmp\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.182482 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-root\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.182520 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-root\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.182561 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.182593 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-tls\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.182618 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-textfile\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.183065 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:15:11.183103 2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 13:15:11.183271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.183115 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-sys\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.184049 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:15:11.183172 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-tls podName:982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d nodeName:}" failed. No retries permitted until 2026-07-07 13:15:11.683153449 +0000 UTC m=+106.737575089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-tls") pod "node-exporter-mz4st" (UID: "982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d") : secret "node-exporter-tls" not found Jul 07 13:15:11.184049 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.183403 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-textfile\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.184049 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.183527 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-metrics-client-ca\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.186211 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.186141 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.193236 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.193167 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlw29\" (UniqueName: \"kubernetes.io/projected/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-kube-api-access-nlw29\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.689106 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.689058 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-tls\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.691614 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.691582 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d-node-exporter-tls\") pod \"node-exporter-mz4st\" (UID: \"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d\") " pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:11.899277 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:11.899219 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mz4st" Jul 07 13:15:13.510065 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:13.510025 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:15:15.173198 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:15:15.173160 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod982ce6f0_c8a6_4ac9_bd2e_b71cceb7e37d.slice/crio-5a3e786b2736c8902e0c8e164a2ff89fb876e88ed0dbe5f6a52df7d5ef8c037e WatchSource:0}: Error finding container 5a3e786b2736c8902e0c8e164a2ff89fb876e88ed0dbe5f6a52df7d5ef8c037e: Status 404 returned error can't find the container with id 5a3e786b2736c8902e0c8e164a2ff89fb876e88ed0dbe5f6a52df7d5ef8c037e Jul 07 13:15:15.944232 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:15.943279 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-2jh4q" event={"ID":"4efdac5c-6234-4954-9fe4-5b2d5dcb2303","Type":"ContainerStarted","Data":"8499aebbf426deb18cc1e3c8c2bb51c0119b18f78afe1306f4c234d7b8d21597"} Jul 07 13:15:15.944232 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:15.944067 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-2jh4q" Jul 07 13:15:15.946892 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:15.946860 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mccwm" event={"ID":"e144ab3c-4492-409c-9ac1-06fad984629b","Type":"ContainerStarted","Data":"e44774215cba2e6be9858e6061208462142fc65498e53602f00197159d177de6"} Jul 07 13:15:15.949738 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:15.949696 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r6464" event={"ID":"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d","Type":"ContainerStarted","Data":"76bf6346e6ebaec7fbd051a1fa02f10d680756c4574faa77ca33f981750e659e"} Jul 07 13:15:15.949911 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:15.949768 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r6464" event={"ID":"aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d","Type":"ContainerStarted","Data":"55bd3a6951e72ec70c9c65e933581bf57e49ffaaf6fee2d66bd05c71e92889b8"} Jul 07 13:15:15.950572 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:15.950505 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r6464" Jul 07 13:15:15.952223 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:15.952155 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mz4st" event={"ID":"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d","Type":"ContainerStarted","Data":"5a3e786b2736c8902e0c8e164a2ff89fb876e88ed0dbe5f6a52df7d5ef8c037e"} Jul 07 13:15:15.958328 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:15.958201 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-2jh4q" Jul 07 13:15:15.966343 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:15.966145 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-2jh4q" podStartSLOduration=1.547306114 podStartE2EDuration="22.966119176s" totalImagesPullingTime="21.418813062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:15.964390753 +0000 UTC m=+111.018812444" watchObservedRunningTime="2026-07-07 13:15:15.966119176 +0000 UTC m=+111.020540836" Jul 07 13:15:15.973258 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:15.973206 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-87b889985-54znm"] Jul 07 13:15:16.005513 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:16.004063 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-r6464" podStartSLOduration=65.659559571 podStartE2EDuration="1m16.004038974s" totalImagesPullingTime="10.344479403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:16.004012251 +0000 UTC m=+111.058433926" watchObservedRunningTime="2026-07-07 13:15:16.004038974 +0000 UTC m=+111.058460638" Jul 07 13:15:16.036091 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:16.035959 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mccwm" podStartSLOduration=34.319954332 podStartE2EDuration="56.035934964s" totalImagesPullingTime="21.715980632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:14:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:16.033892095 +0000 UTC m=+111.088313779" watchObservedRunningTime="2026-07-07 13:15:16.035934964 +0000 UTC m=+111.090356623" Jul 07 13:15:17.963994 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:17.963958 2574 generic.go:356] "Generic (PLEG): container finished" podID="982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d" containerID="c86e4d825361f7f3f9a7b131de63b66fbd1407b5833aa5492dd79a13d1051366" exitCode=0 Jul 07 13:15:17.964546 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:17.964068 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mz4st" event={"ID":"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d","Type":"ContainerDied","Data":"c86e4d825361f7f3f9a7b131de63b66fbd1407b5833aa5492dd79a13d1051366"} Jul 07 13:15:18.970820 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:18.970780 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mz4st" event={"ID":"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d","Type":"ContainerStarted","Data":"8cb98503200ef3dc72b6d1fdb78337f3d9c8d2791725616e712a68a9d488db32"} Jul 07 13:15:18.971327 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:18.970843 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mz4st" event={"ID":"982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d","Type":"ContainerStarted","Data":"ac716214234cde7e8a79d7625e8ea250e422a6dbcb3911df90cdbfd39c834182"} Jul 07 13:15:18.996909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:18.996844 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mz4st" podStartSLOduration=7.216648786 podStartE2EDuration="8.996821328s" totalImagesPullingTime="1.663977532s" totalInitContainerRuntime="116.19501ms" isStatefulPod=true podCreationTimestamp="2026-07-07 13:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:18.995373398 +0000 UTC m=+114.049795077" watchObservedRunningTime="2026-07-07 13:15:18.996821328 +0000 UTC m=+114.051242987" Jul 07 13:15:25.979070 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:25.979036 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:15:26.967082 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:26.967045 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r6464" Jul 07 13:15:35.307889 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.307840 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:15:35.309326 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.309240 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7978659b89-5cb4w" podUID="006691fe-1d4c-41fd-82a8-740043a25930" containerName="console" containerID="cri-o://87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da" gracePeriod=15 Jul 07 13:15:35.310538 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.310509 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb5fd3-beb1-426e-b2e0-6defab925a6a-metrics-certs\") pod \"network-metrics-daemon-gcblz\" (UID: \"fefb5fd3-beb1-426e-b2e0-6defab925a6a\") " pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:15:35.550192 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.550165 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fjddr\"" Jul 07 13:15:35.556049 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.556016 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gcblz" Jul 07 13:15:35.599899 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.599866 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:15:35.610065 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.610040 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-serving-cert\") pod \"006691fe-1d4c-41fd-82a8-740043a25930\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " Jul 07 13:15:35.610207 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.610083 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-oauth-serving-cert\") pod \"006691fe-1d4c-41fd-82a8-740043a25930\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " Jul 07 13:15:35.610207 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.610103 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-service-ca\") pod \"006691fe-1d4c-41fd-82a8-740043a25930\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " Jul 07 13:15:35.610207 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.610128 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-console-config\") pod \"006691fe-1d4c-41fd-82a8-740043a25930\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " Jul 07 13:15:35.610207 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.610175 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dv6j\" (UniqueName: \"kubernetes.io/projected/006691fe-1d4c-41fd-82a8-740043a25930-kube-api-access-8dv6j\") pod \"006691fe-1d4c-41fd-82a8-740043a25930\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " Jul 07 13:15:35.610207 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.610205 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-oauth-config\") pod \"006691fe-1d4c-41fd-82a8-740043a25930\" (UID: \"006691fe-1d4c-41fd-82a8-740043a25930\") " Jul 07 13:15:35.610588 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.610552 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-console-config" (OuterVolumeSpecName: "console-config") pod "006691fe-1d4c-41fd-82a8-740043a25930" (UID: "006691fe-1d4c-41fd-82a8-740043a25930"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:35.610741 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.610561 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "006691fe-1d4c-41fd-82a8-740043a25930" (UID: "006691fe-1d4c-41fd-82a8-740043a25930"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:35.610852 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.610823 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-service-ca" (OuterVolumeSpecName: "service-ca") pod "006691fe-1d4c-41fd-82a8-740043a25930" (UID: "006691fe-1d4c-41fd-82a8-740043a25930"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:35.612555 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.612527 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "006691fe-1d4c-41fd-82a8-740043a25930" (UID: "006691fe-1d4c-41fd-82a8-740043a25930"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:15:35.612799 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.612778 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "006691fe-1d4c-41fd-82a8-740043a25930" (UID: "006691fe-1d4c-41fd-82a8-740043a25930"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:15:35.612871 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.612805 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/006691fe-1d4c-41fd-82a8-740043a25930-kube-api-access-8dv6j" (OuterVolumeSpecName: "kube-api-access-8dv6j") pod "006691fe-1d4c-41fd-82a8-740043a25930" (UID: "006691fe-1d4c-41fd-82a8-740043a25930"). InnerVolumeSpecName "kube-api-access-8dv6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:35.710746 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.710697 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-serving-cert\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:35.710746 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.710733 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-oauth-serving-cert\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:35.710746 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.710746 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:35.710746 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.710758 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/006691fe-1d4c-41fd-82a8-740043a25930-console-config\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:35.711006 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.710769 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dv6j\" (UniqueName: \"kubernetes.io/projected/006691fe-1d4c-41fd-82a8-740043a25930-kube-api-access-8dv6j\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:35.711006 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.710781 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/006691fe-1d4c-41fd-82a8-740043a25930-console-oauth-config\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:35.716328 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:35.716211 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gcblz"] Jul 07 13:15:35.718942 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:15:35.718914 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfefb5fd3_beb1_426e_b2e0_6defab925a6a.slice/crio-22cc18da15b95427fdacbb75fa2510a5b23f3826a1d58793b993211c5bfe01fa WatchSource:0}: Error finding container 22cc18da15b95427fdacbb75fa2510a5b23f3826a1d58793b993211c5bfe01fa: Status 404 returned error can't find the container with id 22cc18da15b95427fdacbb75fa2510a5b23f3826a1d58793b993211c5bfe01fa Jul 07 13:15:36.038734 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.038696 2574 generic.go:356] "Generic (PLEG): container finished" podID="006691fe-1d4c-41fd-82a8-740043a25930" containerID="87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da" exitCode=0 Jul 07 13:15:36.038948 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.038743 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7978659b89-5cb4w" Jul 07 13:15:36.038948 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.038792 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7978659b89-5cb4w" event={"ID":"006691fe-1d4c-41fd-82a8-740043a25930","Type":"ContainerDied","Data":"87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da"} Jul 07 13:15:36.038948 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.038845 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7978659b89-5cb4w" event={"ID":"006691fe-1d4c-41fd-82a8-740043a25930","Type":"ContainerDied","Data":"092d6d5c91db321fa93416aff1adb6be2fdec8412c35749625ac0baf8edd8955"} Jul 07 13:15:36.038948 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.038885 2574 scope.go:117] "RemoveContainer" containerID="87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da" Jul 07 13:15:36.040228 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.040206 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gcblz" event={"ID":"fefb5fd3-beb1-426e-b2e0-6defab925a6a","Type":"ContainerStarted","Data":"22cc18da15b95427fdacbb75fa2510a5b23f3826a1d58793b993211c5bfe01fa"} Jul 07 13:15:36.048020 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.048000 2574 scope.go:117] "RemoveContainer" containerID="87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da" Jul 07 13:15:36.048349 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:15:36.048329 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da\": container with ID starting with 87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da not found: ID does not exist" containerID="87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da" Jul 07 13:15:36.048427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.048363 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da"} err="failed to get container status \"87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da\": rpc error: code = NotFound desc = could not find container \"87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da\": container with ID starting with 87f61979ebd08515e248a30e0fee50d6f11c24add6535a7e7681aaaa8e4d72da not found: ID does not exist" Jul 07 13:15:36.093602 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.093560 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7978659b89-5cb4w"] Jul 07 13:15:36.115992 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.115948 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7978659b89-5cb4w"] Jul 07 13:15:36.793905 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.793877 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58547f567f-h6ff4_6d6726a9-eb0f-45db-b9e9-5068cbda8413/router/0.log" Jul 07 13:15:36.817540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:36.817514 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d4zmg_64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf/serve-healthcheck-canary/0.log" Jul 07 13:15:37.432686 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:37.432644 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="006691fe-1d4c-41fd-82a8-740043a25930" path="/var/lib/kubelet/pods/006691fe-1d4c-41fd-82a8-740043a25930/volumes" Jul 07 13:15:39.057218 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:39.057174 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gcblz" event={"ID":"fefb5fd3-beb1-426e-b2e0-6defab925a6a","Type":"ContainerStarted","Data":"382298ecad8a6cf33343f0df5b3b39638717517cd7ae69635feee4db0e280e78"} Jul 07 13:15:39.057660 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:39.057308 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gcblz" event={"ID":"fefb5fd3-beb1-426e-b2e0-6defab925a6a","Type":"ContainerStarted","Data":"f0537edae5fd399680e0b17225426e6c6e993885646c16b5e5d5e4338aad63ee"} Jul 07 13:15:39.100770 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:39.100638 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gcblz" podStartSLOduration=130.857226536 podStartE2EDuration="2m13.100614531s" totalImagesPullingTime="2.243387995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:39.097900307 +0000 UTC m=+134.152321965" watchObservedRunningTime="2026-07-07 13:15:39.100614531 +0000 UTC m=+134.155036190" Jul 07 13:15:41.004186 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.004136 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-87b889985-54znm" podUID="8beaeed0-113a-49fc-84b9-8a1e16193878" containerName="registry" containerID="cri-o://03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19" gracePeriod=30 Jul 07 13:15:41.281312 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.281275 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:15:41.359936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.359895 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-bound-sa-token\") pod \"8beaeed0-113a-49fc-84b9-8a1e16193878\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " Jul 07 13:15:41.359936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.359940 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") pod \"8beaeed0-113a-49fc-84b9-8a1e16193878\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " Jul 07 13:15:41.360227 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.359997 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8beaeed0-113a-49fc-84b9-8a1e16193878-ca-trust-extracted\") pod \"8beaeed0-113a-49fc-84b9-8a1e16193878\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " Jul 07 13:15:41.360227 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.360084 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5nm6\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-kube-api-access-g5nm6\") pod \"8beaeed0-113a-49fc-84b9-8a1e16193878\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " Jul 07 13:15:41.360227 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.360131 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-trusted-ca\") pod \"8beaeed0-113a-49fc-84b9-8a1e16193878\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " Jul 07 13:15:41.360227 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.360160 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-image-registry-private-configuration\") pod \"8beaeed0-113a-49fc-84b9-8a1e16193878\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " Jul 07 13:15:41.360227 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.360191 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-certificates\") pod \"8beaeed0-113a-49fc-84b9-8a1e16193878\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " Jul 07 13:15:41.360513 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.360232 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-installation-pull-secrets\") pod \"8beaeed0-113a-49fc-84b9-8a1e16193878\" (UID: \"8beaeed0-113a-49fc-84b9-8a1e16193878\") " Jul 07 13:15:41.360816 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.360785 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8beaeed0-113a-49fc-84b9-8a1e16193878" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:41.360926 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.360861 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8beaeed0-113a-49fc-84b9-8a1e16193878" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:41.362651 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.362618 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8beaeed0-113a-49fc-84b9-8a1e16193878" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:41.362776 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.362685 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8beaeed0-113a-49fc-84b9-8a1e16193878" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:15:41.363073 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.363044 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-kube-api-access-g5nm6" (OuterVolumeSpecName: "kube-api-access-g5nm6") pod "8beaeed0-113a-49fc-84b9-8a1e16193878" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878"). InnerVolumeSpecName "kube-api-access-g5nm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:41.363209 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.363073 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8beaeed0-113a-49fc-84b9-8a1e16193878" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:15:41.363209 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.363110 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8beaeed0-113a-49fc-84b9-8a1e16193878" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:41.368958 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.368929 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8beaeed0-113a-49fc-84b9-8a1e16193878-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8beaeed0-113a-49fc-84b9-8a1e16193878" (UID: "8beaeed0-113a-49fc-84b9-8a1e16193878"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:41.461415 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.461376 2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8beaeed0-113a-49fc-84b9-8a1e16193878-ca-trust-extracted\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:41.461415 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.461409 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5nm6\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-kube-api-access-g5nm6\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:41.461415 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.461425 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-trusted-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:41.461725 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.461439 2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-image-registry-private-configuration\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:41.461725 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.461453 2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-certificates\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:41.461725 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.461465 2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8beaeed0-113a-49fc-84b9-8a1e16193878-installation-pull-secrets\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:41.461725 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.461476 2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-bound-sa-token\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:41.461725 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:41.461487 2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8beaeed0-113a-49fc-84b9-8a1e16193878-registry-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:15:42.069151 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:42.069113 2574 generic.go:356] "Generic (PLEG): container finished" podID="8beaeed0-113a-49fc-84b9-8a1e16193878" containerID="03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19" exitCode=0 Jul 07 13:15:42.069680 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:42.069169 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87b889985-54znm" Jul 07 13:15:42.069680 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:42.069205 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87b889985-54znm" event={"ID":"8beaeed0-113a-49fc-84b9-8a1e16193878","Type":"ContainerDied","Data":"03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19"} Jul 07 13:15:42.069680 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:42.069291 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87b889985-54znm" event={"ID":"8beaeed0-113a-49fc-84b9-8a1e16193878","Type":"ContainerDied","Data":"7d4d06f88b7ca8091044fafbd9e879879a7a2dcf9cf787c0475fab8230dadbbb"} Jul 07 13:15:42.069680 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:42.069322 2574 scope.go:117] "RemoveContainer" containerID="03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19" Jul 07 13:15:42.077883 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:42.077860 2574 scope.go:117] "RemoveContainer" containerID="03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19" Jul 07 13:15:42.078237 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:15:42.078210 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19\": container with ID starting with 03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19 not found: ID does not exist" containerID="03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19" Jul 07 13:15:42.078343 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:42.078262 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19"} err="failed to get container status \"03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19\": rpc error: code = NotFound desc = could not find container \"03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19\": container with ID starting with 03cbed8cf68cd41649efd4278a5ba6184d27db24d26666be97f5fae4e53e9e19 not found: ID does not exist" Jul 07 13:15:42.088713 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:42.088681 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-87b889985-54znm"] Jul 07 13:15:42.092893 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:42.092867 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-87b889985-54znm"] Jul 07 13:15:43.431152 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:43.431112 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8beaeed0-113a-49fc-84b9-8a1e16193878" path="/var/lib/kubelet/pods/8beaeed0-113a-49fc-84b9-8a1e16193878/volumes" Jul 07 13:15:44.078765 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:44.078728 2574 generic.go:356] "Generic (PLEG): container finished" podID="7f569c81-186b-46b5-a6a3-7d10df78af9f" containerID="118f15bc6e4553e15c3b8b59adc90b91b9fe30df343cfd3070f15ce49b6e3559" exitCode=0 Jul 07 13:15:44.078916 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:44.078801 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" event={"ID":"7f569c81-186b-46b5-a6a3-7d10df78af9f","Type":"ContainerDied","Data":"118f15bc6e4553e15c3b8b59adc90b91b9fe30df343cfd3070f15ce49b6e3559"} Jul 07 13:15:44.079200 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:44.079181 2574 scope.go:117] "RemoveContainer" containerID="118f15bc6e4553e15c3b8b59adc90b91b9fe30df343cfd3070f15ce49b6e3559" Jul 07 13:15:45.083894 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:45.083861 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-g4q8r" event={"ID":"7f569c81-186b-46b5-a6a3-7d10df78af9f","Type":"ContainerStarted","Data":"f4afbd4efcf1654de46a06c3d3c22dacfa5f9df8ca3e09b2673d21e597638485"} Jul 07 13:15:57.131020 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:57.130980 2574 generic.go:356] "Generic (PLEG): container finished" podID="1137478a-31b6-40e6-a2d3-c7222e4680f1" containerID="1557be1e3e46e98d50624e1165fef3303aed4a651bbef30dbe191a94d04af26c" exitCode=0 Jul 07 13:15:57.131511 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:57.131055 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" event={"ID":"1137478a-31b6-40e6-a2d3-c7222e4680f1","Type":"ContainerDied","Data":"1557be1e3e46e98d50624e1165fef3303aed4a651bbef30dbe191a94d04af26c"} Jul 07 13:15:57.131511 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:57.131386 2574 scope.go:117] "RemoveContainer" containerID="1557be1e3e46e98d50624e1165fef3303aed4a651bbef30dbe191a94d04af26c" Jul 07 13:15:58.136586 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:15:58.136547 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-69fmn" event={"ID":"1137478a-31b6-40e6-a2d3-c7222e4680f1","Type":"ContainerStarted","Data":"07a09a064712dcdf85fc7d4c5d8e921fb1223a956f8b1cf2af31b63b99c8d49e"} Jul 07 13:16:39.860612 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.860574 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58f565598-mhdtn"] Jul 07 13:16:39.861139 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.860835 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8beaeed0-113a-49fc-84b9-8a1e16193878" containerName="registry" Jul 07 13:16:39.861139 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.860848 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8beaeed0-113a-49fc-84b9-8a1e16193878" containerName="registry" Jul 07 13:16:39.861139 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.860865 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="006691fe-1d4c-41fd-82a8-740043a25930" containerName="console" Jul 07 13:16:39.861139 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.860870 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="006691fe-1d4c-41fd-82a8-740043a25930" containerName="console" Jul 07 13:16:39.861139 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.860914 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="006691fe-1d4c-41fd-82a8-740043a25930" containerName="console" Jul 07 13:16:39.861139 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.860922 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8beaeed0-113a-49fc-84b9-8a1e16193878" containerName="registry" Jul 07 13:16:39.863488 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.863430 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:39.874941 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.874915 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 13:16:39.875148 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.875130 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 13:16:39.875200 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.875133 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 13:16:39.886111 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.886081 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dkwl9\"" Jul 07 13:16:39.886520 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.886502 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 13:16:39.886739 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.886723 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 13:16:39.886938 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.886922 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 13:16:39.921428 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:39.921396 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f565598-mhdtn"] Jul 07 13:16:40.005494 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.005458 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-trusted-ca-bundle\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.005494 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.005503 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-oauth-serving-cert\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.005724 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.005546 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-console-config\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.005724 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.005571 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-serving-cert\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.005724 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.005588 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-oauth-config\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.005724 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.005622 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-service-ca\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.005724 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.005647 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh4r9\" (UniqueName: \"kubernetes.io/projected/af511fc4-b2ae-4a48-994b-b393ca680c89-kube-api-access-qh4r9\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.106339 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.106295 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-oauth-serving-cert\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.106536 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.106352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-console-config\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.106536 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.106377 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-serving-cert\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.106536 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.106395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-oauth-config\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.106536 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.106431 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-service-ca\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.106536 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.106459 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh4r9\" (UniqueName: \"kubernetes.io/projected/af511fc4-b2ae-4a48-994b-b393ca680c89-kube-api-access-qh4r9\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.106536 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.106498 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-trusted-ca-bundle\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.107289 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.107231 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-service-ca\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.107488 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.107233 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-oauth-serving-cert\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.107488 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.107233 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-console-config\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.107616 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.107395 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-trusted-ca-bundle\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.108960 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.108940 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-oauth-config\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.109153 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.109132 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-serving-cert\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.115090 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.115021 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh4r9\" (UniqueName: \"kubernetes.io/projected/af511fc4-b2ae-4a48-994b-b393ca680c89-kube-api-access-qh4r9\") pod \"console-58f565598-mhdtn\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.172865 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.172823 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:40.313980 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:40.313935 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58f565598-mhdtn"] Jul 07 13:16:40.318290 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:16:40.318239 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf511fc4_b2ae_4a48_994b_b393ca680c89.slice/crio-3870a3cda613ad0507ce57cf24596c12eb7e74b1e29dbbb4a310b09c1c701e61 WatchSource:0}: Error finding container 3870a3cda613ad0507ce57cf24596c12eb7e74b1e29dbbb4a310b09c1c701e61: Status 404 returned error can't find the container with id 3870a3cda613ad0507ce57cf24596c12eb7e74b1e29dbbb4a310b09c1c701e61 Jul 07 13:16:41.303335 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:41.303295 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f565598-mhdtn" event={"ID":"af511fc4-b2ae-4a48-994b-b393ca680c89","Type":"ContainerStarted","Data":"9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca"} Jul 07 13:16:41.303746 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:41.303348 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f565598-mhdtn" event={"ID":"af511fc4-b2ae-4a48-994b-b393ca680c89","Type":"ContainerStarted","Data":"3870a3cda613ad0507ce57cf24596c12eb7e74b1e29dbbb4a310b09c1c701e61"} Jul 07 13:16:41.322882 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:41.322809 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58f565598-mhdtn" podStartSLOduration=2.322786162 podStartE2EDuration="2.322786162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:16:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:41.322552417 +0000 UTC m=+196.376974088" watchObservedRunningTime="2026-07-07 13:16:41.322786162 +0000 UTC m=+196.377207820" Jul 07 13:16:50.173688 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.173640 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:50.174198 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.173710 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:50.178835 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.178805 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:50.340962 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.340932 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:16:50.686883 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.686842 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5db558966b-kpjg4"] Jul 07 13:16:50.689090 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.689069 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.727199 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.727151 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db558966b-kpjg4"] Jul 07 13:16:50.788380 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.788328 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xppxm\" (UniqueName: \"kubernetes.io/projected/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-kube-api-access-xppxm\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.788380 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.788387 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-oauth-serving-cert\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.788628 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.788437 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-service-ca\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.788628 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.788459 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-trusted-ca-bundle\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.788628 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.788520 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-serving-cert\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.788628 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.788570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-oauth-config\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.788628 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.788589 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-config\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.889202 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.889160 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-serving-cert\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.889407 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.889206 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-oauth-config\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.889407 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.889237 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-config\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.889407 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.889303 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xppxm\" (UniqueName: \"kubernetes.io/projected/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-kube-api-access-xppxm\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.889407 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.889334 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-oauth-serving-cert\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.889407 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.889385 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-service-ca\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.889407 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.889409 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-trusted-ca-bundle\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.890137 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.890105 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-oauth-serving-cert\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.890225 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.890136 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-config\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.890225 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.890116 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-service-ca\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.890626 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.890605 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-trusted-ca-bundle\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.891762 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.891738 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-oauth-config\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.891885 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.891867 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-serving-cert\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.916328 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.916296 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xppxm\" (UniqueName: \"kubernetes.io/projected/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-kube-api-access-xppxm\") pod \"console-5db558966b-kpjg4\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:50.999374 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:50.999337 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:16:51.141739 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:51.141642 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db558966b-kpjg4"] Jul 07 13:16:51.143984 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:16:51.143953 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd09eb8_d752_4a3a_ac75_25f4ee2da02b.slice/crio-6e6147f05e4bd9235cc62e72adaeb55d9721ec2ffc31ee8e0c7f2b863ebb956a WatchSource:0}: Error finding container 6e6147f05e4bd9235cc62e72adaeb55d9721ec2ffc31ee8e0c7f2b863ebb956a: Status 404 returned error can't find the container with id 6e6147f05e4bd9235cc62e72adaeb55d9721ec2ffc31ee8e0c7f2b863ebb956a Jul 07 13:16:51.343262 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:51.343151 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db558966b-kpjg4" event={"ID":"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b","Type":"ContainerStarted","Data":"b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5"} Jul 07 13:16:51.343262 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:51.343198 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db558966b-kpjg4" event={"ID":"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b","Type":"ContainerStarted","Data":"6e6147f05e4bd9235cc62e72adaeb55d9721ec2ffc31ee8e0c7f2b863ebb956a"} Jul 07 13:16:51.369584 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:16:51.369530 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5db558966b-kpjg4" podStartSLOduration=1.369510049 podStartE2EDuration="1.369510049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:16:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:51.36874013 +0000 UTC m=+206.423161810" watchObservedRunningTime="2026-07-07 13:16:51.369510049 +0000 UTC m=+206.423931706" Jul 07 13:17:01.000455 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:01.000395 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:17:01.001006 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:01.000486 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:17:01.005427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:01.005396 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:17:01.383058 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:01.382976 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:17:01.464311 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:01.464273 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f565598-mhdtn"] Jul 07 13:17:26.490878 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.490812 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-58f565598-mhdtn" podUID="af511fc4-b2ae-4a48-994b-b393ca680c89" containerName="console" containerID="cri-o://9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca" gracePeriod=15 Jul 07 13:17:26.743395 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.743318 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:17:26.879428 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.879376 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-trusted-ca-bundle\") pod \"af511fc4-b2ae-4a48-994b-b393ca680c89\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " Jul 07 13:17:26.879428 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.879430 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-oauth-config\") pod \"af511fc4-b2ae-4a48-994b-b393ca680c89\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " Jul 07 13:17:26.879694 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.879456 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-serving-cert\") pod \"af511fc4-b2ae-4a48-994b-b393ca680c89\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " Jul 07 13:17:26.879694 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.879494 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-service-ca\") pod \"af511fc4-b2ae-4a48-994b-b393ca680c89\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " Jul 07 13:17:26.879694 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.879538 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh4r9\" (UniqueName: \"kubernetes.io/projected/af511fc4-b2ae-4a48-994b-b393ca680c89-kube-api-access-qh4r9\") pod \"af511fc4-b2ae-4a48-994b-b393ca680c89\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " Jul 07 13:17:26.879694 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.879567 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-console-config\") pod \"af511fc4-b2ae-4a48-994b-b393ca680c89\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " Jul 07 13:17:26.879694 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.879598 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-oauth-serving-cert\") pod \"af511fc4-b2ae-4a48-994b-b393ca680c89\" (UID: \"af511fc4-b2ae-4a48-994b-b393ca680c89\") " Jul 07 13:17:26.879995 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.879961 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "af511fc4-b2ae-4a48-994b-b393ca680c89" (UID: "af511fc4-b2ae-4a48-994b-b393ca680c89"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:17:26.880060 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.880023 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-service-ca" (OuterVolumeSpecName: "service-ca") pod "af511fc4-b2ae-4a48-994b-b393ca680c89" (UID: "af511fc4-b2ae-4a48-994b-b393ca680c89"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:17:26.880060 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.880034 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "af511fc4-b2ae-4a48-994b-b393ca680c89" (UID: "af511fc4-b2ae-4a48-994b-b393ca680c89"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:17:26.880188 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.880164 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-console-config" (OuterVolumeSpecName: "console-config") pod "af511fc4-b2ae-4a48-994b-b393ca680c89" (UID: "af511fc4-b2ae-4a48-994b-b393ca680c89"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:17:26.881758 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.881725 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "af511fc4-b2ae-4a48-994b-b393ca680c89" (UID: "af511fc4-b2ae-4a48-994b-b393ca680c89"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:17:26.881758 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.881745 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af511fc4-b2ae-4a48-994b-b393ca680c89-kube-api-access-qh4r9" (OuterVolumeSpecName: "kube-api-access-qh4r9") pod "af511fc4-b2ae-4a48-994b-b393ca680c89" (UID: "af511fc4-b2ae-4a48-994b-b393ca680c89"). InnerVolumeSpecName "kube-api-access-qh4r9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:17:26.881758 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.881744 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "af511fc4-b2ae-4a48-994b-b393ca680c89" (UID: "af511fc4-b2ae-4a48-994b-b393ca680c89"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:17:26.980968 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.980924 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:17:26.980968 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.980959 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh4r9\" (UniqueName: \"kubernetes.io/projected/af511fc4-b2ae-4a48-994b-b393ca680c89-kube-api-access-qh4r9\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:17:26.980968 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.980971 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-console-config\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:17:26.980968 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.980980 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-oauth-serving-cert\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:17:26.981243 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.980989 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af511fc4-b2ae-4a48-994b-b393ca680c89-trusted-ca-bundle\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:17:26.981243 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.980996 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-oauth-config\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:17:26.981243 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:26.981005 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af511fc4-b2ae-4a48-994b-b393ca680c89-console-serving-cert\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:17:27.471593 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:27.471558 2574 generic.go:356] "Generic (PLEG): container finished" podID="af511fc4-b2ae-4a48-994b-b393ca680c89" containerID="9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca" exitCode=0 Jul 07 13:17:27.471772 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:27.471603 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58f565598-mhdtn" Jul 07 13:17:27.471772 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:27.471624 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f565598-mhdtn" event={"ID":"af511fc4-b2ae-4a48-994b-b393ca680c89","Type":"ContainerDied","Data":"9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca"} Jul 07 13:17:27.471772 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:27.471662 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58f565598-mhdtn" event={"ID":"af511fc4-b2ae-4a48-994b-b393ca680c89","Type":"ContainerDied","Data":"3870a3cda613ad0507ce57cf24596c12eb7e74b1e29dbbb4a310b09c1c701e61"} Jul 07 13:17:27.471772 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:27.471682 2574 scope.go:117] "RemoveContainer" containerID="9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca" Jul 07 13:17:27.479828 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:27.479809 2574 scope.go:117] "RemoveContainer" containerID="9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca" Jul 07 13:17:27.480159 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:17:27.480137 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca\": container with ID starting with 9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca not found: ID does not exist" containerID="9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca" Jul 07 13:17:27.480232 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:27.480171 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca"} err="failed to get container status \"9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca\": rpc error: code = NotFound desc = could not find container \"9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca\": container with ID starting with 9439eb28f1890fa1962d7f3e649d27eb5a8eb40a7ea06f94ccecbbc2027b2eca not found: ID does not exist" Jul 07 13:17:27.488731 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:27.488695 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58f565598-mhdtn"] Jul 07 13:17:27.493061 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:27.493029 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58f565598-mhdtn"] Jul 07 13:17:29.430784 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:17:29.430746 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af511fc4-b2ae-4a48-994b-b393ca680c89" path="/var/lib/kubelet/pods/af511fc4-b2ae-4a48-994b-b393ca680c89/volumes" Jul 07 13:18:04.972442 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:04.972397 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp"] Jul 07 13:18:04.973185 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:04.972780 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af511fc4-b2ae-4a48-994b-b393ca680c89" containerName="console" Jul 07 13:18:04.973185 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:04.972797 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af511fc4-b2ae-4a48-994b-b393ca680c89" containerName="console" Jul 07 13:18:04.973185 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:04.972869 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="af511fc4-b2ae-4a48-994b-b393ca680c89" containerName="console" Jul 07 13:18:04.975107 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:04.975081 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:04.981740 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:04.981708 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v7pgb\"" Jul 07 13:18:04.981740 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:04.981737 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 13:18:04.981974 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:04.981762 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 13:18:04.987462 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:04.987430 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp"] Jul 07 13:18:05.082029 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.081984 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:05.082271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.082038 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bllzv\" (UniqueName: \"kubernetes.io/projected/686d6956-4af6-4f8a-b48c-0f150889ad9d-kube-api-access-bllzv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:05.082271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.082093 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:05.183451 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.183396 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:05.183451 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.183451 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bllzv\" (UniqueName: \"kubernetes.io/projected/686d6956-4af6-4f8a-b48c-0f150889ad9d-kube-api-access-bllzv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:05.183748 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.183478 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:05.183857 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.183838 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:05.183921 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.183904 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:05.193339 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.193306 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bllzv\" (UniqueName: \"kubernetes.io/projected/686d6956-4af6-4f8a-b48c-0f150889ad9d-kube-api-access-bllzv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:05.290827 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.290731 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:05.430984 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.430961 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp"] Jul 07 13:18:05.433549 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:18:05.433519 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod686d6956_4af6_4f8a_b48c_0f150889ad9d.slice/crio-6dcbe713f1aff08b37b67b3bd682900553456f1a308fd30f5de9a8bb0d82b068 WatchSource:0}: Error finding container 6dcbe713f1aff08b37b67b3bd682900553456f1a308fd30f5de9a8bb0d82b068: Status 404 returned error can't find the container with id 6dcbe713f1aff08b37b67b3bd682900553456f1a308fd30f5de9a8bb0d82b068 Jul 07 13:18:05.616835 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:05.616743 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" event={"ID":"686d6956-4af6-4f8a-b48c-0f150889ad9d","Type":"ContainerStarted","Data":"6dcbe713f1aff08b37b67b3bd682900553456f1a308fd30f5de9a8bb0d82b068"} Jul 07 13:18:12.648134 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:12.648095 2574 generic.go:356] "Generic (PLEG): container finished" podID="686d6956-4af6-4f8a-b48c-0f150889ad9d" containerID="6f1015f9bb3106317003435da47749b4360def066226ca64c5fc6b5bb903bb98" exitCode=0 Jul 07 13:18:12.648555 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:12.648181 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" event={"ID":"686d6956-4af6-4f8a-b48c-0f150889ad9d","Type":"ContainerDied","Data":"6f1015f9bb3106317003435da47749b4360def066226ca64c5fc6b5bb903bb98"} Jul 07 13:18:14.659057 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:14.659015 2574 generic.go:356] "Generic (PLEG): container finished" podID="686d6956-4af6-4f8a-b48c-0f150889ad9d" containerID="d141842432741a58e43450225a51ea2048055146444ad32e8e7c62fa4bd94bae" exitCode=0 Jul 07 13:18:14.659498 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:14.659061 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" event={"ID":"686d6956-4af6-4f8a-b48c-0f150889ad9d","Type":"ContainerDied","Data":"d141842432741a58e43450225a51ea2048055146444ad32e8e7c62fa4bd94bae"} Jul 07 13:18:22.692958 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:22.692916 2574 generic.go:356] "Generic (PLEG): container finished" podID="686d6956-4af6-4f8a-b48c-0f150889ad9d" containerID="95aac90fe704f569e319202caf59bfa4b99aec23e1c5b35b0d0653b8590a0a61" exitCode=0 Jul 07 13:18:22.693399 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:22.693010 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" event={"ID":"686d6956-4af6-4f8a-b48c-0f150889ad9d","Type":"ContainerDied","Data":"95aac90fe704f569e319202caf59bfa4b99aec23e1c5b35b0d0653b8590a0a61"} Jul 07 13:18:23.821542 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:23.821517 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:23.940323 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:23.940276 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-bundle\") pod \"686d6956-4af6-4f8a-b48c-0f150889ad9d\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " Jul 07 13:18:23.940532 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:23.940353 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bllzv\" (UniqueName: \"kubernetes.io/projected/686d6956-4af6-4f8a-b48c-0f150889ad9d-kube-api-access-bllzv\") pod \"686d6956-4af6-4f8a-b48c-0f150889ad9d\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " Jul 07 13:18:23.940532 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:23.940388 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-util\") pod \"686d6956-4af6-4f8a-b48c-0f150889ad9d\" (UID: \"686d6956-4af6-4f8a-b48c-0f150889ad9d\") " Jul 07 13:18:23.941047 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:23.941017 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-bundle" (OuterVolumeSpecName: "bundle") pod "686d6956-4af6-4f8a-b48c-0f150889ad9d" (UID: "686d6956-4af6-4f8a-b48c-0f150889ad9d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:18:23.942592 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:23.942565 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/686d6956-4af6-4f8a-b48c-0f150889ad9d-kube-api-access-bllzv" (OuterVolumeSpecName: "kube-api-access-bllzv") pod "686d6956-4af6-4f8a-b48c-0f150889ad9d" (UID: "686d6956-4af6-4f8a-b48c-0f150889ad9d"). InnerVolumeSpecName "kube-api-access-bllzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:18:23.945033 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:23.945003 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-util" (OuterVolumeSpecName: "util") pod "686d6956-4af6-4f8a-b48c-0f150889ad9d" (UID: "686d6956-4af6-4f8a-b48c-0f150889ad9d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:18:24.040870 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:24.040829 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-bundle\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:18:24.040870 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:24.040862 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bllzv\" (UniqueName: \"kubernetes.io/projected/686d6956-4af6-4f8a-b48c-0f150889ad9d-kube-api-access-bllzv\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:18:24.040870 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:24.040872 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/686d6956-4af6-4f8a-b48c-0f150889ad9d-util\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:18:24.702387 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:24.702354 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" Jul 07 13:18:24.702566 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:24.702360 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ck7sp" event={"ID":"686d6956-4af6-4f8a-b48c-0f150889ad9d","Type":"ContainerDied","Data":"6dcbe713f1aff08b37b67b3bd682900553456f1a308fd30f5de9a8bb0d82b068"} Jul 07 13:18:24.702566 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:24.702468 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6dcbe713f1aff08b37b67b3bd682900553456f1a308fd30f5de9a8bb0d82b068" Jul 07 13:18:25.379370 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:25.379339 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 13:18:28.187516 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.187475 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf"] Jul 07 13:18:28.232507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.187760 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="686d6956-4af6-4f8a-b48c-0f150889ad9d" containerName="extract" Jul 07 13:18:28.232507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.187775 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="686d6956-4af6-4f8a-b48c-0f150889ad9d" containerName="extract" Jul 07 13:18:28.232507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.187787 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="686d6956-4af6-4f8a-b48c-0f150889ad9d" containerName="util" Jul 07 13:18:28.232507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.187793 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="686d6956-4af6-4f8a-b48c-0f150889ad9d" containerName="util" Jul 07 13:18:28.232507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.187799 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="686d6956-4af6-4f8a-b48c-0f150889ad9d" containerName="pull" Jul 07 13:18:28.232507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.187804 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="686d6956-4af6-4f8a-b48c-0f150889ad9d" containerName="pull" Jul 07 13:18:28.232507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.187862 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="686d6956-4af6-4f8a-b48c-0f150889ad9d" containerName="extract" Jul 07 13:18:28.232507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.232481 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf"] Jul 07 13:18:28.232813 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.232580 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf" Jul 07 13:18:28.235802 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.235782 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:18:28.236043 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.236024 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 13:18:28.236137 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.236104 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-c4zjp\"" Jul 07 13:18:28.376486 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.376444 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5p4h\" (UniqueName: \"kubernetes.io/projected/bbb38574-b7cf-455b-9cb4-13bc9e68daa5-kube-api-access-h5p4h\") pod \"cert-manager-operator-controller-manager-55c444bf86-xqfmf\" (UID: \"bbb38574-b7cf-455b-9cb4-13bc9e68daa5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf" Jul 07 13:18:28.376671 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.376574 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbb38574-b7cf-455b-9cb4-13bc9e68daa5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-xqfmf\" (UID: \"bbb38574-b7cf-455b-9cb4-13bc9e68daa5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf" Jul 07 13:18:28.477221 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.477179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbb38574-b7cf-455b-9cb4-13bc9e68daa5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-xqfmf\" (UID: \"bbb38574-b7cf-455b-9cb4-13bc9e68daa5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf" Jul 07 13:18:28.477463 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.477236 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5p4h\" (UniqueName: \"kubernetes.io/projected/bbb38574-b7cf-455b-9cb4-13bc9e68daa5-kube-api-access-h5p4h\") pod \"cert-manager-operator-controller-manager-55c444bf86-xqfmf\" (UID: \"bbb38574-b7cf-455b-9cb4-13bc9e68daa5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf" Jul 07 13:18:28.477666 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.477641 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbb38574-b7cf-455b-9cb4-13bc9e68daa5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-xqfmf\" (UID: \"bbb38574-b7cf-455b-9cb4-13bc9e68daa5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf" Jul 07 13:18:28.486392 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.486357 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5p4h\" (UniqueName: \"kubernetes.io/projected/bbb38574-b7cf-455b-9cb4-13bc9e68daa5-kube-api-access-h5p4h\") pod \"cert-manager-operator-controller-manager-55c444bf86-xqfmf\" (UID: \"bbb38574-b7cf-455b-9cb4-13bc9e68daa5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf" Jul 07 13:18:28.542392 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.542351 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf" Jul 07 13:18:28.761226 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.761192 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf"] Jul 07 13:18:28.764354 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:18:28.764322 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbb38574_b7cf_455b_9cb4_13bc9e68daa5.slice/crio-849c05b26344cf5adf0bbd4dcd5307dad6e11835f2231936fbe4f70edc2c266f WatchSource:0}: Error finding container 849c05b26344cf5adf0bbd4dcd5307dad6e11835f2231936fbe4f70edc2c266f: Status 404 returned error can't find the container with id 849c05b26344cf5adf0bbd4dcd5307dad6e11835f2231936fbe4f70edc2c266f Jul 07 13:18:28.767113 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:28.767093 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:18:29.722763 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:29.722714 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf" event={"ID":"bbb38574-b7cf-455b-9cb4-13bc9e68daa5","Type":"ContainerStarted","Data":"849c05b26344cf5adf0bbd4dcd5307dad6e11835f2231936fbe4f70edc2c266f"} Jul 07 13:18:31.732512 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:31.732466 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf" event={"ID":"bbb38574-b7cf-455b-9cb4-13bc9e68daa5","Type":"ContainerStarted","Data":"1d4cb00ce9d04c09ce7bc62886e38afbcafded0b51941ecc3d3db35ffcb8044e"} Jul 07 13:18:31.890168 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:31.890110 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xqfmf" podStartSLOduration=1.6698652200000001 podStartE2EDuration="3.890092445s" totalImagesPullingTime="2.220227225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:18:31.889137494 +0000 UTC m=+306.943559177" watchObservedRunningTime="2026-07-07 13:18:31.890092445 +0000 UTC m=+306.944514103" Jul 07 13:18:32.566021 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.565983 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946"] Jul 07 13:18:32.588448 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.588420 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:32.592177 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.592149 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 13:18:32.592453 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.592427 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 13:18:32.592505 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.592494 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v7pgb\"" Jul 07 13:18:32.606853 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.606823 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946"] Jul 07 13:18:32.714671 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.714633 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn5vb\" (UniqueName: \"kubernetes.io/projected/4a59d613-cce0-4dfe-9e00-278b402306fd-kube-api-access-kn5vb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:32.714671 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.714675 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:32.714911 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.714697 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:32.815924 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.815881 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn5vb\" (UniqueName: \"kubernetes.io/projected/4a59d613-cce0-4dfe-9e00-278b402306fd-kube-api-access-kn5vb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:32.815924 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.815927 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:32.815924 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.815945 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:32.816527 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.816376 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:32.816527 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.816434 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:32.836019 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.835982 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn5vb\" (UniqueName: \"kubernetes.io/projected/4a59d613-cce0-4dfe-9e00-278b402306fd-kube-api-access-kn5vb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:32.897797 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:32.897756 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:33.041003 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:33.040973 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946"] Jul 07 13:18:33.043647 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:18:33.043616 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a59d613_cce0_4dfe_9e00_278b402306fd.slice/crio-5d61678bd23dfb1d1b8b41bf7be48a4f394845fedc8ef6c082c5a494b6dd6cd2 WatchSource:0}: Error finding container 5d61678bd23dfb1d1b8b41bf7be48a4f394845fedc8ef6c082c5a494b6dd6cd2: Status 404 returned error can't find the container with id 5d61678bd23dfb1d1b8b41bf7be48a4f394845fedc8ef6c082c5a494b6dd6cd2 Jul 07 13:18:33.742625 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:33.742584 2574 generic.go:356] "Generic (PLEG): container finished" podID="4a59d613-cce0-4dfe-9e00-278b402306fd" containerID="07feb48121933483675fbba908cfd2fb68e19aa5069cb05e51eda69625cda817" exitCode=0 Jul 07 13:18:33.742824 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:33.742673 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" event={"ID":"4a59d613-cce0-4dfe-9e00-278b402306fd","Type":"ContainerDied","Data":"07feb48121933483675fbba908cfd2fb68e19aa5069cb05e51eda69625cda817"} Jul 07 13:18:33.742824 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:33.742734 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" event={"ID":"4a59d613-cce0-4dfe-9e00-278b402306fd","Type":"ContainerStarted","Data":"5d61678bd23dfb1d1b8b41bf7be48a4f394845fedc8ef6c082c5a494b6dd6cd2"} Jul 07 13:18:36.756672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:36.756633 2574 generic.go:356] "Generic (PLEG): container finished" podID="4a59d613-cce0-4dfe-9e00-278b402306fd" containerID="ef47437191171c8fdd72f9c406b5743b502ba8513413211eda38c120587abb08" exitCode=0 Jul 07 13:18:36.757112 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:36.756753 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" event={"ID":"4a59d613-cce0-4dfe-9e00-278b402306fd","Type":"ContainerDied","Data":"ef47437191171c8fdd72f9c406b5743b502ba8513413211eda38c120587abb08"} Jul 07 13:18:37.763376 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:37.763335 2574 generic.go:356] "Generic (PLEG): container finished" podID="4a59d613-cce0-4dfe-9e00-278b402306fd" containerID="b7886a87bc037bac9a526c77f75e10327c91755c7a0399bca2516f812531549b" exitCode=0 Jul 07 13:18:37.763796 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:37.763385 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" event={"ID":"4a59d613-cce0-4dfe-9e00-278b402306fd","Type":"ContainerDied","Data":"b7886a87bc037bac9a526c77f75e10327c91755c7a0399bca2516f812531549b"} Jul 07 13:18:38.891164 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:38.891140 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:38.971449 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:38.971410 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kn5vb\" (UniqueName: \"kubernetes.io/projected/4a59d613-cce0-4dfe-9e00-278b402306fd-kube-api-access-kn5vb\") pod \"4a59d613-cce0-4dfe-9e00-278b402306fd\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " Jul 07 13:18:38.971625 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:38.971477 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-bundle\") pod \"4a59d613-cce0-4dfe-9e00-278b402306fd\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " Jul 07 13:18:38.971625 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:38.971569 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-util\") pod \"4a59d613-cce0-4dfe-9e00-278b402306fd\" (UID: \"4a59d613-cce0-4dfe-9e00-278b402306fd\") " Jul 07 13:18:38.971917 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:38.971882 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-bundle" (OuterVolumeSpecName: "bundle") pod "4a59d613-cce0-4dfe-9e00-278b402306fd" (UID: "4a59d613-cce0-4dfe-9e00-278b402306fd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:18:38.973702 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:38.973668 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a59d613-cce0-4dfe-9e00-278b402306fd-kube-api-access-kn5vb" (OuterVolumeSpecName: "kube-api-access-kn5vb") pod "4a59d613-cce0-4dfe-9e00-278b402306fd" (UID: "4a59d613-cce0-4dfe-9e00-278b402306fd"). InnerVolumeSpecName "kube-api-access-kn5vb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:18:38.976422 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:38.976383 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-util" (OuterVolumeSpecName: "util") pod "4a59d613-cce0-4dfe-9e00-278b402306fd" (UID: "4a59d613-cce0-4dfe-9e00-278b402306fd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:18:39.072367 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:39.072233 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-bundle\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:18:39.072367 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:39.072300 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a59d613-cce0-4dfe-9e00-278b402306fd-util\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:18:39.072367 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:39.072309 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn5vb\" (UniqueName: \"kubernetes.io/projected/4a59d613-cce0-4dfe-9e00-278b402306fd-kube-api-access-kn5vb\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:18:39.774359 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:39.774325 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" Jul 07 13:18:39.774359 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:39.774333 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpv946" event={"ID":"4a59d613-cce0-4dfe-9e00-278b402306fd","Type":"ContainerDied","Data":"5d61678bd23dfb1d1b8b41bf7be48a4f394845fedc8ef6c082c5a494b6dd6cd2"} Jul 07 13:18:39.774574 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:39.774379 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d61678bd23dfb1d1b8b41bf7be48a4f394845fedc8ef6c082c5a494b6dd6cd2" Jul 07 13:18:42.323798 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.323759 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-74hqs"] Jul 07 13:18:42.324210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.324082 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a59d613-cce0-4dfe-9e00-278b402306fd" containerName="pull" Jul 07 13:18:42.324210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.324094 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a59d613-cce0-4dfe-9e00-278b402306fd" containerName="pull" Jul 07 13:18:42.324210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.324105 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a59d613-cce0-4dfe-9e00-278b402306fd" containerName="extract" Jul 07 13:18:42.324210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.324110 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a59d613-cce0-4dfe-9e00-278b402306fd" containerName="extract" Jul 07 13:18:42.324210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.324117 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a59d613-cce0-4dfe-9e00-278b402306fd" containerName="util" Jul 07 13:18:42.324210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.324123 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a59d613-cce0-4dfe-9e00-278b402306fd" containerName="util" Jul 07 13:18:42.324210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.324168 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a59d613-cce0-4dfe-9e00-278b402306fd" containerName="extract" Jul 07 13:18:42.326034 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.326018 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-74hqs" Jul 07 13:18:42.328444 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.328419 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9qgvb\"" Jul 07 13:18:42.328588 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.328570 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 13:18:42.329312 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.329291 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 13:18:42.338540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.338512 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-74hqs"] Jul 07 13:18:42.502387 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.502339 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8htd\" (UniqueName: \"kubernetes.io/projected/8383cb64-38e8-4b00-ba34-4f9afa1f6da2-kube-api-access-s8htd\") pod \"cert-manager-55b4d954d5-74hqs\" (UID: \"8383cb64-38e8-4b00-ba34-4f9afa1f6da2\") " pod="cert-manager/cert-manager-55b4d954d5-74hqs" Jul 07 13:18:42.502387 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.502398 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8383cb64-38e8-4b00-ba34-4f9afa1f6da2-bound-sa-token\") pod \"cert-manager-55b4d954d5-74hqs\" (UID: \"8383cb64-38e8-4b00-ba34-4f9afa1f6da2\") " pod="cert-manager/cert-manager-55b4d954d5-74hqs" Jul 07 13:18:42.602937 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.602841 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8htd\" (UniqueName: \"kubernetes.io/projected/8383cb64-38e8-4b00-ba34-4f9afa1f6da2-kube-api-access-s8htd\") pod \"cert-manager-55b4d954d5-74hqs\" (UID: \"8383cb64-38e8-4b00-ba34-4f9afa1f6da2\") " pod="cert-manager/cert-manager-55b4d954d5-74hqs" Jul 07 13:18:42.602937 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.602896 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8383cb64-38e8-4b00-ba34-4f9afa1f6da2-bound-sa-token\") pod \"cert-manager-55b4d954d5-74hqs\" (UID: \"8383cb64-38e8-4b00-ba34-4f9afa1f6da2\") " pod="cert-manager/cert-manager-55b4d954d5-74hqs" Jul 07 13:18:42.612953 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.612916 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8383cb64-38e8-4b00-ba34-4f9afa1f6da2-bound-sa-token\") pod \"cert-manager-55b4d954d5-74hqs\" (UID: \"8383cb64-38e8-4b00-ba34-4f9afa1f6da2\") " pod="cert-manager/cert-manager-55b4d954d5-74hqs" Jul 07 13:18:42.613435 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.613412 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8htd\" (UniqueName: \"kubernetes.io/projected/8383cb64-38e8-4b00-ba34-4f9afa1f6da2-kube-api-access-s8htd\") pod \"cert-manager-55b4d954d5-74hqs\" (UID: \"8383cb64-38e8-4b00-ba34-4f9afa1f6da2\") " pod="cert-manager/cert-manager-55b4d954d5-74hqs" Jul 07 13:18:42.635397 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.635354 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-74hqs" Jul 07 13:18:42.790693 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:42.790657 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-74hqs"] Jul 07 13:18:42.794407 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:18:42.794372 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8383cb64_38e8_4b00_ba34_4f9afa1f6da2.slice/crio-6a84c37a359f4a3d1db23a551ba764d5ef50516ceb221ee807d8b46a6f73c5a5 WatchSource:0}: Error finding container 6a84c37a359f4a3d1db23a551ba764d5ef50516ceb221ee807d8b46a6f73c5a5: Status 404 returned error can't find the container with id 6a84c37a359f4a3d1db23a551ba764d5ef50516ceb221ee807d8b46a6f73c5a5 Jul 07 13:18:43.793932 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:43.793889 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-74hqs" event={"ID":"8383cb64-38e8-4b00-ba34-4f9afa1f6da2","Type":"ContainerStarted","Data":"6a84c37a359f4a3d1db23a551ba764d5ef50516ceb221ee807d8b46a6f73c5a5"} Jul 07 13:18:45.804529 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:45.804491 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-74hqs" event={"ID":"8383cb64-38e8-4b00-ba34-4f9afa1f6da2","Type":"ContainerStarted","Data":"c6147f987e6fea9b134b7be833345c3af6e83736f6acf6396d6e627c84675323"} Jul 07 13:18:45.827428 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:45.827368 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-74hqs" podStartSLOduration=1.597165105 podStartE2EDuration="3.827347748s" totalImagesPullingTime="2.230182643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:18:45.826557811 +0000 UTC m=+320.880979472" watchObservedRunningTime="2026-07-07 13:18:45.827347748 +0000 UTC m=+320.881769411" Jul 07 13:18:46.309743 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.309701 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75"] Jul 07 13:18:46.311692 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.311672 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75" Jul 07 13:18:46.314582 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.314549 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-g9bgp\"" Jul 07 13:18:46.315240 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.315221 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 13:18:46.315403 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.315273 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:18:46.323343 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.323316 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75"] Jul 07 13:18:46.435923 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.435879 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67knn\" (UniqueName: \"kubernetes.io/projected/5be6b9dd-ce78-4adc-b14c-f9920f389b0b-kube-api-access-67knn\") pod \"openshift-lws-operator-fd8ccff4c-vxv75\" (UID: \"5be6b9dd-ce78-4adc-b14c-f9920f389b0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75" Jul 07 13:18:46.436097 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.435945 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5be6b9dd-ce78-4adc-b14c-f9920f389b0b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vxv75\" (UID: \"5be6b9dd-ce78-4adc-b14c-f9920f389b0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75" Jul 07 13:18:46.536410 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.536363 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67knn\" (UniqueName: \"kubernetes.io/projected/5be6b9dd-ce78-4adc-b14c-f9920f389b0b-kube-api-access-67knn\") pod \"openshift-lws-operator-fd8ccff4c-vxv75\" (UID: \"5be6b9dd-ce78-4adc-b14c-f9920f389b0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75" Jul 07 13:18:46.536599 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.536464 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5be6b9dd-ce78-4adc-b14c-f9920f389b0b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vxv75\" (UID: \"5be6b9dd-ce78-4adc-b14c-f9920f389b0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75" Jul 07 13:18:46.536800 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.536783 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5be6b9dd-ce78-4adc-b14c-f9920f389b0b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vxv75\" (UID: \"5be6b9dd-ce78-4adc-b14c-f9920f389b0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75" Jul 07 13:18:46.546141 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.546108 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67knn\" (UniqueName: \"kubernetes.io/projected/5be6b9dd-ce78-4adc-b14c-f9920f389b0b-kube-api-access-67knn\") pod \"openshift-lws-operator-fd8ccff4c-vxv75\" (UID: \"5be6b9dd-ce78-4adc-b14c-f9920f389b0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75" Jul 07 13:18:46.621527 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.621421 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75" Jul 07 13:18:46.766776 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.766612 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75"] Jul 07 13:18:46.768925 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:18:46.768896 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5be6b9dd_ce78_4adc_b14c_f9920f389b0b.slice/crio-f5af63f0a90a46c0bcc7ceb9257e1fc833c8115a2bd3fe98d226e092c726d118 WatchSource:0}: Error finding container f5af63f0a90a46c0bcc7ceb9257e1fc833c8115a2bd3fe98d226e092c726d118: Status 404 returned error can't find the container with id f5af63f0a90a46c0bcc7ceb9257e1fc833c8115a2bd3fe98d226e092c726d118 Jul 07 13:18:46.810733 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:46.810691 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75" event={"ID":"5be6b9dd-ce78-4adc-b14c-f9920f389b0b","Type":"ContainerStarted","Data":"f5af63f0a90a46c0bcc7ceb9257e1fc833c8115a2bd3fe98d226e092c726d118"} Jul 07 13:18:49.827325 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:49.827290 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75" event={"ID":"5be6b9dd-ce78-4adc-b14c-f9920f389b0b","Type":"ContainerStarted","Data":"7b7431e2d01724e89b651b2dc9a4b39f60606685d1b0d2c40cea345f33193a46"} Jul 07 13:18:49.850311 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:49.850229 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vxv75" podStartSLOduration=1.813525115 podStartE2EDuration="3.850209566s" totalImagesPullingTime="2.036684451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:18:49.849671343 +0000 UTC m=+324.904093003" watchObservedRunningTime="2026-07-07 13:18:49.850209566 +0000 UTC m=+324.904631224" Jul 07 13:18:52.437790 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.437747 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t"] Jul 07 13:18:52.441172 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.441152 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:52.443960 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.443937 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 13:18:52.444781 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.444763 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 13:18:52.444846 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.444794 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v7pgb\"" Jul 07 13:18:52.451771 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.451745 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t"] Jul 07 13:18:52.488489 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.488451 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:52.488489 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.488490 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:52.488754 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.488527 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4mnn\" (UniqueName: \"kubernetes.io/projected/d4a7af85-189c-4d94-b270-9bd1a3975bec-kube-api-access-x4mnn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:52.589612 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.589576 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:52.589612 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.589612 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:52.589817 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.589648 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4mnn\" (UniqueName: \"kubernetes.io/projected/d4a7af85-189c-4d94-b270-9bd1a3975bec-kube-api-access-x4mnn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:52.590033 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.590010 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:52.590117 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.590046 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:52.598941 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.598914 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4mnn\" (UniqueName: \"kubernetes.io/projected/d4a7af85-189c-4d94-b270-9bd1a3975bec-kube-api-access-x4mnn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:52.750120 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.750080 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:52.890697 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:52.890504 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t"] Jul 07 13:18:52.893540 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:18:52.893500 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a7af85_189c_4d94_b270_9bd1a3975bec.slice/crio-a5209a53a5029026e68c7c1cf4739136b22f483ba868090b578a71efa3e33751 WatchSource:0}: Error finding container a5209a53a5029026e68c7c1cf4739136b22f483ba868090b578a71efa3e33751: Status 404 returned error can't find the container with id a5209a53a5029026e68c7c1cf4739136b22f483ba868090b578a71efa3e33751 Jul 07 13:18:53.847801 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:53.847768 2574 generic.go:356] "Generic (PLEG): container finished" podID="d4a7af85-189c-4d94-b270-9bd1a3975bec" containerID="edf5b2ef2d014d2705f248f42e24885d2eaf30d4646f708d8640deed71604324" exitCode=0 Jul 07 13:18:53.848329 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:53.847813 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" event={"ID":"d4a7af85-189c-4d94-b270-9bd1a3975bec","Type":"ContainerDied","Data":"edf5b2ef2d014d2705f248f42e24885d2eaf30d4646f708d8640deed71604324"} Jul 07 13:18:53.848329 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:53.847850 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" event={"ID":"d4a7af85-189c-4d94-b270-9bd1a3975bec","Type":"ContainerStarted","Data":"a5209a53a5029026e68c7c1cf4739136b22f483ba868090b578a71efa3e33751"} Jul 07 13:18:54.854698 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:54.854665 2574 generic.go:356] "Generic (PLEG): container finished" podID="d4a7af85-189c-4d94-b270-9bd1a3975bec" containerID="b52543cb699735123e38815e6286563e44d521f55308ec56f1e77da8f3ad350f" exitCode=0 Jul 07 13:18:54.855099 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:54.854707 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" event={"ID":"d4a7af85-189c-4d94-b270-9bd1a3975bec","Type":"ContainerDied","Data":"b52543cb699735123e38815e6286563e44d521f55308ec56f1e77da8f3ad350f"} Jul 07 13:18:55.861851 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:55.861811 2574 generic.go:356] "Generic (PLEG): container finished" podID="d4a7af85-189c-4d94-b270-9bd1a3975bec" containerID="0a8c23cda37824514f4996e79d682e3e3b56599babcfd49fb8f09ad38a8bd1e1" exitCode=0 Jul 07 13:18:55.862240 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:55.861861 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" event={"ID":"d4a7af85-189c-4d94-b270-9bd1a3975bec","Type":"ContainerDied","Data":"0a8c23cda37824514f4996e79d682e3e3b56599babcfd49fb8f09ad38a8bd1e1"} Jul 07 13:18:57.006992 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.006956 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:57.021087 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.021048 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4mnn\" (UniqueName: \"kubernetes.io/projected/d4a7af85-189c-4d94-b270-9bd1a3975bec-kube-api-access-x4mnn\") pod \"d4a7af85-189c-4d94-b270-9bd1a3975bec\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " Jul 07 13:18:57.021212 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.021103 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-bundle\") pod \"d4a7af85-189c-4d94-b270-9bd1a3975bec\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " Jul 07 13:18:57.021212 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.021129 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-util\") pod \"d4a7af85-189c-4d94-b270-9bd1a3975bec\" (UID: \"d4a7af85-189c-4d94-b270-9bd1a3975bec\") " Jul 07 13:18:57.021855 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.021826 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-bundle" (OuterVolumeSpecName: "bundle") pod "d4a7af85-189c-4d94-b270-9bd1a3975bec" (UID: "d4a7af85-189c-4d94-b270-9bd1a3975bec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:18:57.023313 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.023281 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a7af85-189c-4d94-b270-9bd1a3975bec-kube-api-access-x4mnn" (OuterVolumeSpecName: "kube-api-access-x4mnn") pod "d4a7af85-189c-4d94-b270-9bd1a3975bec" (UID: "d4a7af85-189c-4d94-b270-9bd1a3975bec"). InnerVolumeSpecName "kube-api-access-x4mnn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:18:57.026953 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.026894 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-util" (OuterVolumeSpecName: "util") pod "d4a7af85-189c-4d94-b270-9bd1a3975bec" (UID: "d4a7af85-189c-4d94-b270-9bd1a3975bec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:18:57.121782 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.121680 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4mnn\" (UniqueName: \"kubernetes.io/projected/d4a7af85-189c-4d94-b270-9bd1a3975bec-kube-api-access-x4mnn\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:18:57.121782 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.121723 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-bundle\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:18:57.121782 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.121732 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4a7af85-189c-4d94-b270-9bd1a3975bec-util\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:18:57.872664 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.872628 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" event={"ID":"d4a7af85-189c-4d94-b270-9bd1a3975bec","Type":"ContainerDied","Data":"a5209a53a5029026e68c7c1cf4739136b22f483ba868090b578a71efa3e33751"} Jul 07 13:18:57.872851 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.872675 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9gk9t" Jul 07 13:18:57.872851 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:18:57.872679 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5209a53a5029026e68c7c1cf4739136b22f483ba868090b578a71efa3e33751" Jul 07 13:19:05.290494 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.290459 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp"] Jul 07 13:19:05.290895 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.290735 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a7af85-189c-4d94-b270-9bd1a3975bec" containerName="pull" Jul 07 13:19:05.290895 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.290745 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a7af85-189c-4d94-b270-9bd1a3975bec" containerName="pull" Jul 07 13:19:05.290895 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.290753 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a7af85-189c-4d94-b270-9bd1a3975bec" containerName="util" Jul 07 13:19:05.290895 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.290759 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a7af85-189c-4d94-b270-9bd1a3975bec" containerName="util" Jul 07 13:19:05.290895 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.290776 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a7af85-189c-4d94-b270-9bd1a3975bec" containerName="extract" Jul 07 13:19:05.290895 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.290782 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a7af85-189c-4d94-b270-9bd1a3975bec" containerName="extract" Jul 07 13:19:05.290895 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.290826 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a7af85-189c-4d94-b270-9bd1a3975bec" containerName="extract" Jul 07 13:19:05.293284 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.293236 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.297427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.297394 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 13:19:05.297567 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.297432 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 13:19:05.297567 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.297446 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 13:19:05.297567 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.297432 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 13:19:05.297745 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.297662 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ssqv6\"" Jul 07 13:19:05.317394 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.317361 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp"] Jul 07 13:19:05.388760 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.388713 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dc832fe3-7c94-4645-b49c-a99bd87a3e11-manifests\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.388971 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.388783 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dc832fe3-7c94-4645-b49c-a99bd87a3e11-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.388971 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.388811 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc832fe3-7c94-4645-b49c-a99bd87a3e11-tmp\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.388971 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.388842 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dc832fe3-7c94-4645-b49c-a99bd87a3e11-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.388971 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.388858 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbnwq\" (UniqueName: \"kubernetes.io/projected/dc832fe3-7c94-4645-b49c-a99bd87a3e11-kube-api-access-vbnwq\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.489571 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.489532 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dc832fe3-7c94-4645-b49c-a99bd87a3e11-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.489571 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.489574 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc832fe3-7c94-4645-b49c-a99bd87a3e11-tmp\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.489829 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.489608 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dc832fe3-7c94-4645-b49c-a99bd87a3e11-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.489829 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.489626 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbnwq\" (UniqueName: \"kubernetes.io/projected/dc832fe3-7c94-4645-b49c-a99bd87a3e11-kube-api-access-vbnwq\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.489829 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.489646 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dc832fe3-7c94-4645-b49c-a99bd87a3e11-manifests\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.490076 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.490048 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dc832fe3-7c94-4645-b49c-a99bd87a3e11-manifests\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.491910 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.491876 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc832fe3-7c94-4645-b49c-a99bd87a3e11-tmp\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.492155 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.492134 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dc832fe3-7c94-4645-b49c-a99bd87a3e11-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.492226 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.492155 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dc832fe3-7c94-4645-b49c-a99bd87a3e11-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.505302 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.505270 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbnwq\" (UniqueName: \"kubernetes.io/projected/dc832fe3-7c94-4645-b49c-a99bd87a3e11-kube-api-access-vbnwq\") pod \"opendatahub-operator-controller-manager-7bdc7d5545-56jhp\" (UID: \"dc832fe3-7c94-4645-b49c-a99bd87a3e11\") " pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.604003 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.603906 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:05.752585 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.752557 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp"] Jul 07 13:19:05.754692 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:19:05.754660 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc832fe3_7c94_4645_b49c_a99bd87a3e11.slice/crio-2b59897d119374fcce657d1abd763332b74ffc0cda1ff34515ce66c08b03cdef WatchSource:0}: Error finding container 2b59897d119374fcce657d1abd763332b74ffc0cda1ff34515ce66c08b03cdef: Status 404 returned error can't find the container with id 2b59897d119374fcce657d1abd763332b74ffc0cda1ff34515ce66c08b03cdef Jul 07 13:19:05.907523 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:05.907432 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" event={"ID":"dc832fe3-7c94-4645-b49c-a99bd87a3e11","Type":"ContainerStarted","Data":"2b59897d119374fcce657d1abd763332b74ffc0cda1ff34515ce66c08b03cdef"} Jul 07 13:19:08.930437 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:08.930392 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" event={"ID":"dc832fe3-7c94-4645-b49c-a99bd87a3e11","Type":"ContainerStarted","Data":"1c5463dca69fab3c7cbb0017125216dcb4eb0ab7f669b2219dc82d58d1bba8ed"} Jul 07 13:19:10.940471 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:10.940435 2574 generic.go:356] "Generic (PLEG): container finished" podID="dc832fe3-7c94-4645-b49c-a99bd87a3e11" containerID="1c5463dca69fab3c7cbb0017125216dcb4eb0ab7f669b2219dc82d58d1bba8ed" exitCode=0 Jul 07 13:19:10.940907 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:10.940504 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" event={"ID":"dc832fe3-7c94-4645-b49c-a99bd87a3e11","Type":"ContainerDied","Data":"1c5463dca69fab3c7cbb0017125216dcb4eb0ab7f669b2219dc82d58d1bba8ed"} Jul 07 13:19:11.947209 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:11.947169 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" event={"ID":"dc832fe3-7c94-4645-b49c-a99bd87a3e11","Type":"ContainerStarted","Data":"3a417a4ef3eace1839076ce2cb4bc776fded7d6f492e7d98f8bdb5b15262ef8e"} Jul 07 13:19:11.947643 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:11.947410 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:12.378950 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:12.378890 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" podStartSLOduration=2.718010754 podStartE2EDuration="7.378869993s" totalImagesPullingTime="3.489474271s" totalInitContainerRuntime="1.171384968s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:12.37493183 +0000 UTC m=+347.429353488" watchObservedRunningTime="2026-07-07 13:19:12.378869993 +0000 UTC m=+347.433291650" Jul 07 13:19:22.953678 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:22.953582 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bdc7d5545-56jhp" Jul 07 13:19:25.208807 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.208768 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5747f4757f-blw6t"] Jul 07 13:19:25.213608 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.213580 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" Jul 07 13:19:25.217088 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.217062 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 13:19:25.217627 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.217599 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vr5k8\"" Jul 07 13:19:25.217815 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.217632 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 13:19:25.225806 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.225781 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5747f4757f-blw6t"] Jul 07 13:19:25.364367 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.364322 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b21a741e-f724-4457-957e-f8162defa8be-tls-certs\") pod \"kube-auth-proxy-5747f4757f-blw6t\" (UID: \"b21a741e-f724-4457-957e-f8162defa8be\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" Jul 07 13:19:25.364569 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.364373 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq85n\" (UniqueName: \"kubernetes.io/projected/b21a741e-f724-4457-957e-f8162defa8be-kube-api-access-kq85n\") pod \"kube-auth-proxy-5747f4757f-blw6t\" (UID: \"b21a741e-f724-4457-957e-f8162defa8be\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" Jul 07 13:19:25.364569 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.364498 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b21a741e-f724-4457-957e-f8162defa8be-tmp\") pod \"kube-auth-proxy-5747f4757f-blw6t\" (UID: \"b21a741e-f724-4457-957e-f8162defa8be\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" Jul 07 13:19:25.465573 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.465476 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b21a741e-f724-4457-957e-f8162defa8be-tmp\") pod \"kube-auth-proxy-5747f4757f-blw6t\" (UID: \"b21a741e-f724-4457-957e-f8162defa8be\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" Jul 07 13:19:25.465573 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.465519 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b21a741e-f724-4457-957e-f8162defa8be-tls-certs\") pod \"kube-auth-proxy-5747f4757f-blw6t\" (UID: \"b21a741e-f724-4457-957e-f8162defa8be\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" Jul 07 13:19:25.465573 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.465549 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq85n\" (UniqueName: \"kubernetes.io/projected/b21a741e-f724-4457-957e-f8162defa8be-kube-api-access-kq85n\") pod \"kube-auth-proxy-5747f4757f-blw6t\" (UID: \"b21a741e-f724-4457-957e-f8162defa8be\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" Jul 07 13:19:25.468041 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.468007 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b21a741e-f724-4457-957e-f8162defa8be-tmp\") pod \"kube-auth-proxy-5747f4757f-blw6t\" (UID: \"b21a741e-f724-4457-957e-f8162defa8be\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" Jul 07 13:19:25.470895 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.470869 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 13:19:25.478352 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.478324 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq85n\" (UniqueName: \"kubernetes.io/projected/b21a741e-f724-4457-957e-f8162defa8be-kube-api-access-kq85n\") pod \"kube-auth-proxy-5747f4757f-blw6t\" (UID: \"b21a741e-f724-4457-957e-f8162defa8be\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" Jul 07 13:19:25.478495 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.478378 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b21a741e-f724-4457-957e-f8162defa8be-tls-certs\") pod \"kube-auth-proxy-5747f4757f-blw6t\" (UID: \"b21a741e-f724-4457-957e-f8162defa8be\") " pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" Jul 07 13:19:25.532436 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.532391 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vr5k8\"" Jul 07 13:19:25.533418 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.533389 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" Jul 07 13:19:25.699584 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:25.699549 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5747f4757f-blw6t"] Jul 07 13:19:25.702849 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:19:25.702812 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb21a741e_f724_4457_957e_f8162defa8be.slice/crio-eb444fd07950209cbd5274ab0343d3925a958ca74cfdb0ce1c3217ca718dc87e WatchSource:0}: Error finding container eb444fd07950209cbd5274ab0343d3925a958ca74cfdb0ce1c3217ca718dc87e: Status 404 returned error can't find the container with id eb444fd07950209cbd5274ab0343d3925a958ca74cfdb0ce1c3217ca718dc87e Jul 07 13:19:26.014288 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.014238 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" event={"ID":"b21a741e-f724-4457-957e-f8162defa8be","Type":"ContainerStarted","Data":"eb444fd07950209cbd5274ab0343d3925a958ca74cfdb0ce1c3217ca718dc87e"} Jul 07 13:19:26.830487 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.830455 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm"] Jul 07 13:19:26.833885 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.833864 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:26.838422 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.838388 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 13:19:26.838649 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.838630 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 13:19:26.838718 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.838673 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gnzkr\"" Jul 07 13:19:26.838767 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.838724 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 13:19:26.838818 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.838804 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 13:19:26.854194 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.854156 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm"] Jul 07 13:19:26.979392 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.979359 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:26.979609 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.979437 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-token\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:26.979609 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.979462 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b96c12a-793c-4b27-9c27-61e0c9632da0-local-certs\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:26.979609 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.979495 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-cacerts\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:26.979784 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.979646 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:26.979784 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.979694 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjjhh\" (UniqueName: \"kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-kube-api-access-cjjhh\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:26.979784 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:26.979722 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.080364 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.080321 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.080605 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.080382 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-token\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.080605 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.080553 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b96c12a-793c-4b27-9c27-61e0c9632da0-local-certs\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.080742 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.080602 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-cacerts\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.080742 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.080678 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.080742 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.080718 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjjhh\" (UniqueName: \"kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-kube-api-access-cjjhh\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.080956 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.080753 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.081367 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.081343 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.082875 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.082845 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.082970 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.082882 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.083153 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.083135 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b96c12a-793c-4b27-9c27-61e0c9632da0-local-certs\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.083299 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.083283 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-cacerts\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.092412 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.092386 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjjhh\" (UniqueName: \"kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-kube-api-access-cjjhh\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.094020 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.094000 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-token\") pod \"istiod-openshift-gateway-7574459656-jc4hm\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.144156 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.144119 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:27.318883 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:27.318681 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm"] Jul 07 13:19:27.321522 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:19:27.321485 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b96c12a_793c_4b27_9c27_61e0c9632da0.slice/crio-e07e8d4078473512ca0a389d98ff314f879509ed4fc616c8e936f76a23ab90ce WatchSource:0}: Error finding container e07e8d4078473512ca0a389d98ff314f879509ed4fc616c8e936f76a23ab90ce: Status 404 returned error can't find the container with id e07e8d4078473512ca0a389d98ff314f879509ed4fc616c8e936f76a23ab90ce Jul 07 13:19:28.025261 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:28.025202 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" event={"ID":"1b96c12a-793c-4b27-9c27-61e0c9632da0","Type":"ContainerStarted","Data":"e07e8d4078473512ca0a389d98ff314f879509ed4fc616c8e936f76a23ab90ce"} Jul 07 13:19:30.354207 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:30.354175 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 13:19:30.404527 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:30.404481 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:19:30.404619 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:30.404559 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:19:31.042894 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:31.042850 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" event={"ID":"1b96c12a-793c-4b27-9c27-61e0c9632da0","Type":"ContainerStarted","Data":"2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa"} Jul 07 13:19:31.043126 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:31.042961 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:31.044727 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:31.044692 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" event={"ID":"b21a741e-f724-4457-957e-f8162defa8be","Type":"ContainerStarted","Data":"89fce127829ee80712278c7e4467cf7d128a597eaacd38f30c276c824b7b26e4"} Jul 07 13:19:31.044944 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:31.044922 2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-jc4hm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 13:19:31.045035 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:31.044973 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" podUID="1b96c12a-793c-4b27-9c27-61e0c9632da0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 13:19:31.066835 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:31.066768 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" podStartSLOduration=1.986487074 podStartE2EDuration="5.066744371s" totalImagesPullingTime="3.080257297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:31.064909306 +0000 UTC m=+366.119330983" watchObservedRunningTime="2026-07-07 13:19:31.066744371 +0000 UTC m=+366.121166030" Jul 07 13:19:31.083783 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:31.083723 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5747f4757f-blw6t" podStartSLOduration=1.439048068 podStartE2EDuration="6.083704717s" totalImagesPullingTime="4.644656649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:31.082452637 +0000 UTC m=+366.136874309" watchObservedRunningTime="2026-07-07 13:19:31.083704717 +0000 UTC m=+366.138126375" Jul 07 13:19:32.050105 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:32.050073 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:34.695984 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.695942 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv"] Jul 07 13:19:34.699538 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.699516 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.719223 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.719190 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv"] Jul 07 13:19:34.854701 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.854657 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.854873 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.854717 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.854873 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.854816 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bkqf\" (UniqueName: \"kubernetes.io/projected/78b96415-a199-4f0d-99ee-24aa1a8f879f-kube-api-access-7bkqf\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.854953 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.854887 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78b96415-a199-4f0d-99ee-24aa1a8f879f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.854953 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.854914 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.854953 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.854938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.855048 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.854995 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78b96415-a199-4f0d-99ee-24aa1a8f879f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.956044 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.956006 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.956272 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.956069 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bkqf\" (UniqueName: \"kubernetes.io/projected/78b96415-a199-4f0d-99ee-24aa1a8f879f-kube-api-access-7bkqf\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.956272 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.956126 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78b96415-a199-4f0d-99ee-24aa1a8f879f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.956272 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.956152 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.956272 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.956181 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.956498 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.956302 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78b96415-a199-4f0d-99ee-24aa1a8f879f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.956498 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.956362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.956911 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.956886 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.958832 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.958807 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.958942 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.958833 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78b96415-a199-4f0d-99ee-24aa1a8f879f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.958990 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.958951 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78b96415-a199-4f0d-99ee-24aa1a8f879f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.959026 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.959001 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.969523 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.969493 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78b96415-a199-4f0d-99ee-24aa1a8f879f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:34.980376 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:34.980340 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bkqf\" (UniqueName: \"kubernetes.io/projected/78b96415-a199-4f0d-99ee-24aa1a8f879f-kube-api-access-7bkqf\") pod \"istiod-openshift-gateway-94bb8fbfd-r4qsv\" (UID: \"78b96415-a199-4f0d-99ee-24aa1a8f879f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:35.009386 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:35.009346 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:35.346748 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:19:35.346707 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78b96415_a199_4f0d_99ee_24aa1a8f879f.slice/crio-df31bd7515f02bccb6fc6751afdad30e2f86577bf2798162791a8bd491ed5630 WatchSource:0}: Error finding container df31bd7515f02bccb6fc6751afdad30e2f86577bf2798162791a8bd491ed5630: Status 404 returned error can't find the container with id df31bd7515f02bccb6fc6751afdad30e2f86577bf2798162791a8bd491ed5630 Jul 07 13:19:35.349164 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:35.349115 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:19:35.349291 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:35.349199 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 13:19:35.352626 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:35.352597 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv"] Jul 07 13:19:36.073609 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:36.073555 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" event={"ID":"78b96415-a199-4f0d-99ee-24aa1a8f879f","Type":"ContainerStarted","Data":"130d0b63026a704a01bceb28a1d3d598bd4320734f318ce34f73cb386f8866fa"} Jul 07 13:19:36.074155 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:36.073633 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" event={"ID":"78b96415-a199-4f0d-99ee-24aa1a8f879f","Type":"ContainerStarted","Data":"df31bd7515f02bccb6fc6751afdad30e2f86577bf2798162791a8bd491ed5630"} Jul 07 13:19:36.074576 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:36.074552 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:36.076178 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:36.076142 2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-r4qsv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 13:19:36.076315 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:36.076205 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" podUID="78b96415-a199-4f0d-99ee-24aa1a8f879f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 13:19:36.108859 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:36.108791 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" podStartSLOduration=2.108765605 podStartE2EDuration="2.108765605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:36.107772111 +0000 UTC m=+371.162193790" watchObservedRunningTime="2026-07-07 13:19:36.108765605 +0000 UTC m=+371.163187264" Jul 07 13:19:37.079217 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.079186 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-r4qsv" Jul 07 13:19:37.219801 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.219756 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm"] Jul 07 13:19:37.220030 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.220005 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" podUID="1b96c12a-793c-4b27-9c27-61e0c9632da0" containerName="discovery" containerID="cri-o://2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa" gracePeriod=30 Jul 07 13:19:37.482371 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.482338 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:37.585002 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.584965 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-kubeconfig\") pod \"1b96c12a-793c-4b27-9c27-61e0c9632da0\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " Jul 07 13:19:37.585181 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.585012 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-token\") pod \"1b96c12a-793c-4b27-9c27-61e0c9632da0\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " Jul 07 13:19:37.585181 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.585031 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-cacerts\") pod \"1b96c12a-793c-4b27-9c27-61e0c9632da0\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " Jul 07 13:19:37.585181 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.585052 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjjhh\" (UniqueName: \"kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-kube-api-access-cjjhh\") pod \"1b96c12a-793c-4b27-9c27-61e0c9632da0\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " Jul 07 13:19:37.585181 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.585101 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b96c12a-793c-4b27-9c27-61e0c9632da0-local-certs\") pod \"1b96c12a-793c-4b27-9c27-61e0c9632da0\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " Jul 07 13:19:37.585181 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.585118 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-dns-cert\") pod \"1b96c12a-793c-4b27-9c27-61e0c9632da0\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " Jul 07 13:19:37.585181 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.585152 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-ca-configmap\") pod \"1b96c12a-793c-4b27-9c27-61e0c9632da0\" (UID: \"1b96c12a-793c-4b27-9c27-61e0c9632da0\") " Jul 07 13:19:37.585631 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.585605 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "1b96c12a-793c-4b27-9c27-61e0c9632da0" (UID: "1b96c12a-793c-4b27-9c27-61e0c9632da0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:19:37.587701 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.587660 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-kube-api-access-cjjhh" (OuterVolumeSpecName: "kube-api-access-cjjhh") pod "1b96c12a-793c-4b27-9c27-61e0c9632da0" (UID: "1b96c12a-793c-4b27-9c27-61e0c9632da0"). InnerVolumeSpecName "kube-api-access-cjjhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:19:37.587701 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.587660 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b96c12a-793c-4b27-9c27-61e0c9632da0" (UID: "1b96c12a-793c-4b27-9c27-61e0c9632da0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:19:37.587876 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.587757 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "1b96c12a-793c-4b27-9c27-61e0c9632da0" (UID: "1b96c12a-793c-4b27-9c27-61e0c9632da0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:19:37.587876 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.587776 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b96c12a-793c-4b27-9c27-61e0c9632da0-local-certs" (OuterVolumeSpecName: "local-certs") pod "1b96c12a-793c-4b27-9c27-61e0c9632da0" (UID: "1b96c12a-793c-4b27-9c27-61e0c9632da0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:19:37.587876 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.587795 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-cacerts" (OuterVolumeSpecName: "cacerts") pod "1b96c12a-793c-4b27-9c27-61e0c9632da0" (UID: "1b96c12a-793c-4b27-9c27-61e0c9632da0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:19:37.587876 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.587809 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "1b96c12a-793c-4b27-9c27-61e0c9632da0" (UID: "1b96c12a-793c-4b27-9c27-61e0c9632da0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:19:37.686271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.686204 2574 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-kubeconfig\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:19:37.686271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.686238 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-token\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:19:37.686271 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.686278 2574 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-cacerts\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:19:37.686524 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.686287 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjjhh\" (UniqueName: \"kubernetes.io/projected/1b96c12a-793c-4b27-9c27-61e0c9632da0-kube-api-access-cjjhh\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:19:37.686524 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.686296 2574 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b96c12a-793c-4b27-9c27-61e0c9632da0-local-certs\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:19:37.686524 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.686305 2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-dns-cert\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:19:37.686524 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:37.686314 2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b96c12a-793c-4b27-9c27-61e0c9632da0-istio-csr-ca-configmap\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:19:38.084728 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:38.084671 2574 generic.go:356] "Generic (PLEG): container finished" podID="1b96c12a-793c-4b27-9c27-61e0c9632da0" containerID="2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa" exitCode=0 Jul 07 13:19:38.084728 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:38.084724 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" Jul 07 13:19:38.085299 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:38.084775 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" event={"ID":"1b96c12a-793c-4b27-9c27-61e0c9632da0","Type":"ContainerDied","Data":"2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa"} Jul 07 13:19:38.085299 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:38.084842 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm" event={"ID":"1b96c12a-793c-4b27-9c27-61e0c9632da0","Type":"ContainerDied","Data":"e07e8d4078473512ca0a389d98ff314f879509ed4fc616c8e936f76a23ab90ce"} Jul 07 13:19:38.085299 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:38.084873 2574 scope.go:117] "RemoveContainer" containerID="2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa" Jul 07 13:19:38.095421 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:38.095395 2574 scope.go:117] "RemoveContainer" containerID="2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa" Jul 07 13:19:38.095780 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:19:38.095758 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa\": container with ID starting with 2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa not found: ID does not exist" containerID="2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa" Jul 07 13:19:38.095888 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:38.095794 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa"} err="failed to get container status \"2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa\": rpc error: code = NotFound desc = could not find container \"2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa\": container with ID starting with 2f5a10efee73e3d48f1625dd9d9afe36dcb47c728f5180c1d0d8c584d0a03afa not found: ID does not exist" Jul 07 13:19:38.131552 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:38.131516 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm"] Jul 07 13:19:38.135869 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:38.135841 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jc4hm"] Jul 07 13:19:39.431749 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:19:39.431715 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b96c12a-793c-4b27-9c27-61e0c9632da0" path="/var/lib/kubelet/pods/1b96c12a-793c-4b27-9c27-61e0c9632da0/volumes" Jul 07 13:20:26.448020 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.447981 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pm2fp"] Jul 07 13:20:26.448479 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.448304 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b96c12a-793c-4b27-9c27-61e0c9632da0" containerName="discovery" Jul 07 13:20:26.448479 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.448319 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b96c12a-793c-4b27-9c27-61e0c9632da0" containerName="discovery" Jul 07 13:20:26.448479 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.448388 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b96c12a-793c-4b27-9c27-61e0c9632da0" containerName="discovery" Jul 07 13:20:26.452536 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.452511 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" Jul 07 13:20:26.457352 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.457327 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 13:20:26.457512 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.457330 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 13:20:26.458176 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.458160 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-q5p8h\"" Jul 07 13:20:26.475642 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.475596 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pm2fp"] Jul 07 13:20:26.602982 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.602945 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2tm6\" (UniqueName: \"kubernetes.io/projected/f60bb43e-52fb-4cf8-8519-7007e7e9d88f-kube-api-access-j2tm6\") pod \"kuadrant-operator-catalog-pm2fp\" (UID: \"f60bb43e-52fb-4cf8-8519-7007e7e9d88f\") " pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" Jul 07 13:20:26.704289 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.704167 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2tm6\" (UniqueName: \"kubernetes.io/projected/f60bb43e-52fb-4cf8-8519-7007e7e9d88f-kube-api-access-j2tm6\") pod \"kuadrant-operator-catalog-pm2fp\" (UID: \"f60bb43e-52fb-4cf8-8519-7007e7e9d88f\") " pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" Jul 07 13:20:26.719207 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.719171 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2tm6\" (UniqueName: \"kubernetes.io/projected/f60bb43e-52fb-4cf8-8519-7007e7e9d88f-kube-api-access-j2tm6\") pod \"kuadrant-operator-catalog-pm2fp\" (UID: \"f60bb43e-52fb-4cf8-8519-7007e7e9d88f\") " pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" Jul 07 13:20:26.762999 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.762947 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" Jul 07 13:20:26.788038 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.788005 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pm2fp"] Jul 07 13:20:26.910552 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.910509 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pm2fp"] Jul 07 13:20:26.912810 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:20:26.912778 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf60bb43e_52fb_4cf8_8519_7007e7e9d88f.slice/crio-3330391582d353ec71b873a848ff0153a98b4113dbb1a0c708bb2b9a370f065c WatchSource:0}: Error finding container 3330391582d353ec71b873a848ff0153a98b4113dbb1a0c708bb2b9a370f065c: Status 404 returned error can't find the container with id 3330391582d353ec71b873a848ff0153a98b4113dbb1a0c708bb2b9a370f065c Jul 07 13:20:26.999222 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:26.999184 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xbpw4"] Jul 07 13:20:27.003511 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:27.003490 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" Jul 07 13:20:27.014119 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:27.014086 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xbpw4"] Jul 07 13:20:27.107998 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:27.107950 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k9h8\" (UniqueName: \"kubernetes.io/projected/84137efc-71b5-4674-9e3f-5c14a442a9f6-kube-api-access-7k9h8\") pod \"kuadrant-operator-catalog-xbpw4\" (UID: \"84137efc-71b5-4674-9e3f-5c14a442a9f6\") " pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" Jul 07 13:20:27.208678 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:27.208641 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k9h8\" (UniqueName: \"kubernetes.io/projected/84137efc-71b5-4674-9e3f-5c14a442a9f6-kube-api-access-7k9h8\") pod \"kuadrant-operator-catalog-xbpw4\" (UID: \"84137efc-71b5-4674-9e3f-5c14a442a9f6\") " pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" Jul 07 13:20:27.218409 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:27.218369 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k9h8\" (UniqueName: \"kubernetes.io/projected/84137efc-71b5-4674-9e3f-5c14a442a9f6-kube-api-access-7k9h8\") pod \"kuadrant-operator-catalog-xbpw4\" (UID: \"84137efc-71b5-4674-9e3f-5c14a442a9f6\") " pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" Jul 07 13:20:27.313952 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:27.313849 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" Jul 07 13:20:27.320841 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:27.320807 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" event={"ID":"f60bb43e-52fb-4cf8-8519-7007e7e9d88f","Type":"ContainerStarted","Data":"3330391582d353ec71b873a848ff0153a98b4113dbb1a0c708bb2b9a370f065c"} Jul 07 13:20:27.460069 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:27.460040 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xbpw4"] Jul 07 13:20:27.462744 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:20:27.462716 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84137efc_71b5_4674_9e3f_5c14a442a9f6.slice/crio-a1b1f3fe4de1cceb97de5d65ca89e716e5018d8fa80106a8890394123bdcd6b9 WatchSource:0}: Error finding container a1b1f3fe4de1cceb97de5d65ca89e716e5018d8fa80106a8890394123bdcd6b9: Status 404 returned error can't find the container with id a1b1f3fe4de1cceb97de5d65ca89e716e5018d8fa80106a8890394123bdcd6b9 Jul 07 13:20:28.329946 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:28.329902 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" event={"ID":"84137efc-71b5-4674-9e3f-5c14a442a9f6","Type":"ContainerStarted","Data":"a1b1f3fe4de1cceb97de5d65ca89e716e5018d8fa80106a8890394123bdcd6b9"} Jul 07 13:20:30.342505 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:30.342468 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" event={"ID":"84137efc-71b5-4674-9e3f-5c14a442a9f6","Type":"ContainerStarted","Data":"f50e1dfe1f096d79066a192caae93a953aa31d24b3cc29ea10b800bdde8c846f"} Jul 07 13:20:30.344102 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:30.344075 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" event={"ID":"f60bb43e-52fb-4cf8-8519-7007e7e9d88f","Type":"ContainerStarted","Data":"3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad"} Jul 07 13:20:30.344204 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:30.344113 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" podUID="f60bb43e-52fb-4cf8-8519-7007e7e9d88f" containerName="registry-server" containerID="cri-o://3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad" gracePeriod=2 Jul 07 13:20:30.365972 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:30.365920 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" podStartSLOduration=2.466824833 podStartE2EDuration="4.365902139s" totalImagesPullingTime="1.899077306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:20:30.364484757 +0000 UTC m=+425.418906416" watchObservedRunningTime="2026-07-07 13:20:30.365902139 +0000 UTC m=+425.420323798" Jul 07 13:20:30.381573 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:30.381517 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" podStartSLOduration=1.9339631449999999 podStartE2EDuration="4.381497351s" totalImagesPullingTime="2.447534206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:20:30.379302284 +0000 UTC m=+425.433723963" watchObservedRunningTime="2026-07-07 13:20:30.381497351 +0000 UTC m=+425.435919009" Jul 07 13:20:30.614979 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:30.614951 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" Jul 07 13:20:30.743560 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:30.743518 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2tm6\" (UniqueName: \"kubernetes.io/projected/f60bb43e-52fb-4cf8-8519-7007e7e9d88f-kube-api-access-j2tm6\") pod \"f60bb43e-52fb-4cf8-8519-7007e7e9d88f\" (UID: \"f60bb43e-52fb-4cf8-8519-7007e7e9d88f\") " Jul 07 13:20:30.745781 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:30.745752 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f60bb43e-52fb-4cf8-8519-7007e7e9d88f-kube-api-access-j2tm6" (OuterVolumeSpecName: "kube-api-access-j2tm6") pod "f60bb43e-52fb-4cf8-8519-7007e7e9d88f" (UID: "f60bb43e-52fb-4cf8-8519-7007e7e9d88f"). InnerVolumeSpecName "kube-api-access-j2tm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:20:30.845079 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:30.845018 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2tm6\" (UniqueName: \"kubernetes.io/projected/f60bb43e-52fb-4cf8-8519-7007e7e9d88f-kube-api-access-j2tm6\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:31.350443 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:31.350408 2574 generic.go:356] "Generic (PLEG): container finished" podID="f60bb43e-52fb-4cf8-8519-7007e7e9d88f" containerID="3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad" exitCode=0 Jul 07 13:20:31.350946 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:31.350450 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" Jul 07 13:20:31.350946 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:31.350500 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" event={"ID":"f60bb43e-52fb-4cf8-8519-7007e7e9d88f","Type":"ContainerDied","Data":"3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad"} Jul 07 13:20:31.350946 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:31.350545 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pm2fp" event={"ID":"f60bb43e-52fb-4cf8-8519-7007e7e9d88f","Type":"ContainerDied","Data":"3330391582d353ec71b873a848ff0153a98b4113dbb1a0c708bb2b9a370f065c"} Jul 07 13:20:31.350946 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:31.350565 2574 scope.go:117] "RemoveContainer" containerID="3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad" Jul 07 13:20:31.360715 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:31.360694 2574 scope.go:117] "RemoveContainer" containerID="3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad" Jul 07 13:20:31.361060 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:20:31.361039 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad\": container with ID starting with 3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad not found: ID does not exist" containerID="3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad" Jul 07 13:20:31.361111 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:31.361069 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad"} err="failed to get container status \"3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad\": rpc error: code = NotFound desc = could not find container \"3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad\": container with ID starting with 3fc9c64261a895846c2d54377d2aaac22bfd2f5d6c100f16503a83ce71f033ad not found: ID does not exist" Jul 07 13:20:31.372861 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:31.372824 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pm2fp"] Jul 07 13:20:31.377321 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:31.377283 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pm2fp"] Jul 07 13:20:31.431892 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:31.431855 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f60bb43e-52fb-4cf8-8519-7007e7e9d88f" path="/var/lib/kubelet/pods/f60bb43e-52fb-4cf8-8519-7007e7e9d88f/volumes" Jul 07 13:20:37.314286 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:37.314221 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" Jul 07 13:20:37.314776 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:37.314309 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" Jul 07 13:20:37.338067 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:37.338036 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" Jul 07 13:20:37.402657 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:37.402627 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xbpw4" Jul 07 13:20:38.252798 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.252731 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk"] Jul 07 13:20:38.253055 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.253042 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f60bb43e-52fb-4cf8-8519-7007e7e9d88f" containerName="registry-server" Jul 07 13:20:38.253097 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.253056 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f60bb43e-52fb-4cf8-8519-7007e7e9d88f" containerName="registry-server" Jul 07 13:20:38.253131 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.253117 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f60bb43e-52fb-4cf8-8519-7007e7e9d88f" containerName="registry-server" Jul 07 13:20:38.257556 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.257531 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:38.260479 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.260452 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7jgsl\"" Jul 07 13:20:38.270216 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.270184 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk"] Jul 07 13:20:38.310975 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.310929 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:38.311168 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.311017 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2448\" (UniqueName: \"kubernetes.io/projected/8500662d-2cd4-4a8c-a6ae-693736363282-kube-api-access-b2448\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:38.311168 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.311066 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:38.411882 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.411812 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:38.412388 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.411950 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2448\" (UniqueName: \"kubernetes.io/projected/8500662d-2cd4-4a8c-a6ae-693736363282-kube-api-access-b2448\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:38.412388 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.411984 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:38.412388 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.412283 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:38.412388 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.412383 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:38.421790 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.421752 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2448\" (UniqueName: \"kubernetes.io/projected/8500662d-2cd4-4a8c-a6ae-693736363282-kube-api-access-b2448\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:38.567121 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.567019 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:38.709089 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.709060 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk"] Jul 07 13:20:38.711903 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:20:38.711867 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8500662d_2cd4_4a8c_a6ae_693736363282.slice/crio-4fc8e0eb91a13da2eb03540b90be930a930f078ccd14fb333880fdcac6c229c7 WatchSource:0}: Error finding container 4fc8e0eb91a13da2eb03540b90be930a930f078ccd14fb333880fdcac6c229c7: Status 404 returned error can't find the container with id 4fc8e0eb91a13da2eb03540b90be930a930f078ccd14fb333880fdcac6c229c7 Jul 07 13:20:38.849459 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.849423 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx"] Jul 07 13:20:38.852835 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.852806 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:38.865482 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.865446 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx"] Jul 07 13:20:38.916086 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.916043 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxrrz\" (UniqueName: \"kubernetes.io/projected/dde178da-32ab-4bc8-b399-556e78e0ef42-kube-api-access-xxrrz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:38.916086 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.916090 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:38.916367 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:38.916111 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:39.016693 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.016657 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxrrz\" (UniqueName: \"kubernetes.io/projected/dde178da-32ab-4bc8-b399-556e78e0ef42-kube-api-access-xxrrz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:39.016693 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.016698 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:39.016932 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.016755 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:39.017148 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.017129 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:39.017183 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.017138 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:39.027370 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.027338 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxrrz\" (UniqueName: \"kubernetes.io/projected/dde178da-32ab-4bc8-b399-556e78e0ef42-kube-api-access-xxrrz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:39.188794 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.188684 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:39.330746 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.330716 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx"] Jul 07 13:20:39.333071 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:20:39.333043 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde178da_32ab_4bc8_b399_556e78e0ef42.slice/crio-687909a01192c4aa031ad54a19252b251a14b2fb1255884b3210c5eb789ad330 WatchSource:0}: Error finding container 687909a01192c4aa031ad54a19252b251a14b2fb1255884b3210c5eb789ad330: Status 404 returned error can't find the container with id 687909a01192c4aa031ad54a19252b251a14b2fb1255884b3210c5eb789ad330 Jul 07 13:20:39.391993 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.391961 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" event={"ID":"dde178da-32ab-4bc8-b399-556e78e0ef42","Type":"ContainerStarted","Data":"687909a01192c4aa031ad54a19252b251a14b2fb1255884b3210c5eb789ad330"} Jul 07 13:20:39.393461 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.393438 2574 generic.go:356] "Generic (PLEG): container finished" podID="8500662d-2cd4-4a8c-a6ae-693736363282" containerID="e5299ff3ead90338e725d5e1c34444fd393ac0d85636ad6d436b5a2441135da8" exitCode=0 Jul 07 13:20:39.393565 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.393515 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" event={"ID":"8500662d-2cd4-4a8c-a6ae-693736363282","Type":"ContainerDied","Data":"e5299ff3ead90338e725d5e1c34444fd393ac0d85636ad6d436b5a2441135da8"} Jul 07 13:20:39.393565 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.393559 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" event={"ID":"8500662d-2cd4-4a8c-a6ae-693736363282","Type":"ContainerStarted","Data":"4fc8e0eb91a13da2eb03540b90be930a930f078ccd14fb333880fdcac6c229c7"} Jul 07 13:20:39.448210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.448174 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb"] Jul 07 13:20:39.451545 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.451525 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:39.463475 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.463434 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb"] Jul 07 13:20:39.520604 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.520567 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:39.520819 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.520635 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:39.520819 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.520693 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzm68\" (UniqueName: \"kubernetes.io/projected/f457658d-bdeb-4d30-a224-6d995404d939-kube-api-access-pzm68\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:39.622102 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.622056 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzm68\" (UniqueName: \"kubernetes.io/projected/f457658d-bdeb-4d30-a224-6d995404d939-kube-api-access-pzm68\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:39.622332 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.622150 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:39.622332 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.622193 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:39.622681 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.622657 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:39.622719 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.622670 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:39.634791 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.634754 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzm68\" (UniqueName: \"kubernetes.io/projected/f457658d-bdeb-4d30-a224-6d995404d939-kube-api-access-pzm68\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:39.804111 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.804063 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:39.962077 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:39.962034 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb"] Jul 07 13:20:39.965743 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:20:39.965705 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf457658d_bdeb_4d30_a224_6d995404d939.slice/crio-41fd432b596381dcb1001f49310db98519365368f09a8b69ef2adce6da897c41 WatchSource:0}: Error finding container 41fd432b596381dcb1001f49310db98519365368f09a8b69ef2adce6da897c41: Status 404 returned error can't find the container with id 41fd432b596381dcb1001f49310db98519365368f09a8b69ef2adce6da897c41 Jul 07 13:20:40.093713 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.093677 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64"] Jul 07 13:20:40.097066 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.097046 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:40.121902 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.121875 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64"] Jul 07 13:20:40.127262 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.127222 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:40.127406 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.127336 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvs5v\" (UniqueName: \"kubernetes.io/projected/017bced3-b77f-407d-b471-00700fd3deac-kube-api-access-lvs5v\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:40.127406 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.127370 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:40.228559 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.228514 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:40.228752 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.228581 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:40.228752 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.228633 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvs5v\" (UniqueName: \"kubernetes.io/projected/017bced3-b77f-407d-b471-00700fd3deac-kube-api-access-lvs5v\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:40.229053 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.229028 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:40.229128 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.229033 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:40.246552 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.246514 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvs5v\" (UniqueName: \"kubernetes.io/projected/017bced3-b77f-407d-b471-00700fd3deac-kube-api-access-lvs5v\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:40.400215 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.400119 2574 generic.go:356] "Generic (PLEG): container finished" podID="f457658d-bdeb-4d30-a224-6d995404d939" containerID="3bda9c7bdd32f91c6385e2253d04a7c15ab769a2051f67735301d7aa9a7418fe" exitCode=0 Jul 07 13:20:40.400397 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.400208 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" event={"ID":"f457658d-bdeb-4d30-a224-6d995404d939","Type":"ContainerDied","Data":"3bda9c7bdd32f91c6385e2253d04a7c15ab769a2051f67735301d7aa9a7418fe"} Jul 07 13:20:40.400397 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.400288 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" event={"ID":"f457658d-bdeb-4d30-a224-6d995404d939","Type":"ContainerStarted","Data":"41fd432b596381dcb1001f49310db98519365368f09a8b69ef2adce6da897c41"} Jul 07 13:20:40.401865 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.401841 2574 generic.go:356] "Generic (PLEG): container finished" podID="dde178da-32ab-4bc8-b399-556e78e0ef42" containerID="9cc529c78d27bd64abe796976e47649a332461a44f6c45822738367327087277" exitCode=0 Jul 07 13:20:40.401865 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.401868 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" event={"ID":"dde178da-32ab-4bc8-b399-556e78e0ef42","Type":"ContainerDied","Data":"9cc529c78d27bd64abe796976e47649a332461a44f6c45822738367327087277"} Jul 07 13:20:40.424070 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.424034 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:40.571893 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:40.571867 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64"] Jul 07 13:20:40.584011 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:20:40.583968 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod017bced3_b77f_407d_b471_00700fd3deac.slice/crio-e99b7d9af5de306e4ef161aea6d2faa07248638263f3f89169e5b4a73b5cfdad WatchSource:0}: Error finding container e99b7d9af5de306e4ef161aea6d2faa07248638263f3f89169e5b4a73b5cfdad: Status 404 returned error can't find the container with id e99b7d9af5de306e4ef161aea6d2faa07248638263f3f89169e5b4a73b5cfdad Jul 07 13:20:41.409182 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:41.409143 2574 generic.go:356] "Generic (PLEG): container finished" podID="8500662d-2cd4-4a8c-a6ae-693736363282" containerID="6e615912c115ede9b013f988ff83bf2c383e1f9b859109a8fa50dc1d7670aa86" exitCode=0 Jul 07 13:20:41.409378 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:41.409233 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" event={"ID":"8500662d-2cd4-4a8c-a6ae-693736363282","Type":"ContainerDied","Data":"6e615912c115ede9b013f988ff83bf2c383e1f9b859109a8fa50dc1d7670aa86"} Jul 07 13:20:41.411092 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:41.411071 2574 generic.go:356] "Generic (PLEG): container finished" podID="017bced3-b77f-407d-b471-00700fd3deac" containerID="5923348a01c25eb03a4b662e7fe3d0e1d599323c1facc240d75f9fac06ebf0c1" exitCode=0 Jul 07 13:20:41.411184 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:41.411157 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" event={"ID":"017bced3-b77f-407d-b471-00700fd3deac","Type":"ContainerDied","Data":"5923348a01c25eb03a4b662e7fe3d0e1d599323c1facc240d75f9fac06ebf0c1"} Jul 07 13:20:41.411267 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:41.411214 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" event={"ID":"017bced3-b77f-407d-b471-00700fd3deac","Type":"ContainerStarted","Data":"e99b7d9af5de306e4ef161aea6d2faa07248638263f3f89169e5b4a73b5cfdad"} Jul 07 13:20:42.418048 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:42.418015 2574 generic.go:356] "Generic (PLEG): container finished" podID="dde178da-32ab-4bc8-b399-556e78e0ef42" containerID="2a9a6c3111f5843059a74bd356bf242179283ae63789c03798ed87cd65f75b4c" exitCode=0 Jul 07 13:20:42.418446 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:42.418096 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" event={"ID":"dde178da-32ab-4bc8-b399-556e78e0ef42","Type":"ContainerDied","Data":"2a9a6c3111f5843059a74bd356bf242179283ae63789c03798ed87cd65f75b4c"} Jul 07 13:20:42.420411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:42.420386 2574 generic.go:356] "Generic (PLEG): container finished" podID="8500662d-2cd4-4a8c-a6ae-693736363282" containerID="dfce00222036be81d3fe28dc28714ec67008262821f1a598a4f444d21d0d7060" exitCode=0 Jul 07 13:20:42.420524 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:42.420505 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" event={"ID":"8500662d-2cd4-4a8c-a6ae-693736363282","Type":"ContainerDied","Data":"dfce00222036be81d3fe28dc28714ec67008262821f1a598a4f444d21d0d7060"} Jul 07 13:20:42.422226 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:42.422202 2574 generic.go:356] "Generic (PLEG): container finished" podID="f457658d-bdeb-4d30-a224-6d995404d939" containerID="fbd88e2df8cb8d19845a78299acdd7448dad26c5385131612c3dd36e3bf543a0" exitCode=0 Jul 07 13:20:42.422341 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:42.422235 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" event={"ID":"f457658d-bdeb-4d30-a224-6d995404d939","Type":"ContainerDied","Data":"fbd88e2df8cb8d19845a78299acdd7448dad26c5385131612c3dd36e3bf543a0"} Jul 07 13:20:43.429578 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.429537 2574 generic.go:356] "Generic (PLEG): container finished" podID="f457658d-bdeb-4d30-a224-6d995404d939" containerID="0185ea1f7f1e75e2b506b391a2047fca11eb72f202ff11093e867575078b82f6" exitCode=0 Jul 07 13:20:43.431640 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.431602 2574 generic.go:356] "Generic (PLEG): container finished" podID="017bced3-b77f-407d-b471-00700fd3deac" containerID="5a05b340a2eb5f130e447640f4da1dd0fe702542d76efc3958e08c2744cc68fc" exitCode=0 Jul 07 13:20:43.432891 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.432692 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" event={"ID":"f457658d-bdeb-4d30-a224-6d995404d939","Type":"ContainerDied","Data":"0185ea1f7f1e75e2b506b391a2047fca11eb72f202ff11093e867575078b82f6"} Jul 07 13:20:43.432891 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.432748 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" event={"ID":"017bced3-b77f-407d-b471-00700fd3deac","Type":"ContainerDied","Data":"5a05b340a2eb5f130e447640f4da1dd0fe702542d76efc3958e08c2744cc68fc"} Jul 07 13:20:43.434161 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.434108 2574 generic.go:356] "Generic (PLEG): container finished" podID="dde178da-32ab-4bc8-b399-556e78e0ef42" containerID="7074dbde7fd5ae3426bf01659f5d70be9ca4cbdb9cde6133a692f3c438301434" exitCode=0 Jul 07 13:20:43.434278 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.434198 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" event={"ID":"dde178da-32ab-4bc8-b399-556e78e0ef42","Type":"ContainerDied","Data":"7074dbde7fd5ae3426bf01659f5d70be9ca4cbdb9cde6133a692f3c438301434"} Jul 07 13:20:43.589373 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.589339 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:43.659612 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.659575 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-util\") pod \"8500662d-2cd4-4a8c-a6ae-693736363282\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " Jul 07 13:20:43.659781 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.659667 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2448\" (UniqueName: \"kubernetes.io/projected/8500662d-2cd4-4a8c-a6ae-693736363282-kube-api-access-b2448\") pod \"8500662d-2cd4-4a8c-a6ae-693736363282\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " Jul 07 13:20:43.659781 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.659702 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-bundle\") pod \"8500662d-2cd4-4a8c-a6ae-693736363282\" (UID: \"8500662d-2cd4-4a8c-a6ae-693736363282\") " Jul 07 13:20:43.660447 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.660410 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-bundle" (OuterVolumeSpecName: "bundle") pod "8500662d-2cd4-4a8c-a6ae-693736363282" (UID: "8500662d-2cd4-4a8c-a6ae-693736363282"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:20:43.661960 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.661932 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8500662d-2cd4-4a8c-a6ae-693736363282-kube-api-access-b2448" (OuterVolumeSpecName: "kube-api-access-b2448") pod "8500662d-2cd4-4a8c-a6ae-693736363282" (UID: "8500662d-2cd4-4a8c-a6ae-693736363282"). InnerVolumeSpecName "kube-api-access-b2448". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:20:43.665812 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.665774 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-util" (OuterVolumeSpecName: "util") pod "8500662d-2cd4-4a8c-a6ae-693736363282" (UID: "8500662d-2cd4-4a8c-a6ae-693736363282"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:20:43.760866 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.760827 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2448\" (UniqueName: \"kubernetes.io/projected/8500662d-2cd4-4a8c-a6ae-693736363282-kube-api-access-b2448\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:43.760866 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.760862 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-bundle\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:43.760866 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:43.760875 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8500662d-2cd4-4a8c-a6ae-693736363282-util\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:44.442524 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.442478 2574 generic.go:356] "Generic (PLEG): container finished" podID="017bced3-b77f-407d-b471-00700fd3deac" containerID="a8c537fa6d73a1ecd1fac63477ad9e5577d76a7022b7a64746246dcc1d575b0c" exitCode=0 Jul 07 13:20:44.442948 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.442547 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" event={"ID":"017bced3-b77f-407d-b471-00700fd3deac","Type":"ContainerDied","Data":"a8c537fa6d73a1ecd1fac63477ad9e5577d76a7022b7a64746246dcc1d575b0c"} Jul 07 13:20:44.444427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.444399 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" event={"ID":"8500662d-2cd4-4a8c-a6ae-693736363282","Type":"ContainerDied","Data":"4fc8e0eb91a13da2eb03540b90be930a930f078ccd14fb333880fdcac6c229c7"} Jul 07 13:20:44.444574 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.444449 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fc8e0eb91a13da2eb03540b90be930a930f078ccd14fb333880fdcac6c229c7" Jul 07 13:20:44.444647 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.444608 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk" Jul 07 13:20:44.602101 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.602073 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:44.605411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.605383 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:44.668394 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.668355 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-util\") pod \"dde178da-32ab-4bc8-b399-556e78e0ef42\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " Jul 07 13:20:44.668596 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.668443 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-bundle\") pod \"dde178da-32ab-4bc8-b399-556e78e0ef42\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " Jul 07 13:20:44.668596 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.668505 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxrrz\" (UniqueName: \"kubernetes.io/projected/dde178da-32ab-4bc8-b399-556e78e0ef42-kube-api-access-xxrrz\") pod \"dde178da-32ab-4bc8-b399-556e78e0ef42\" (UID: \"dde178da-32ab-4bc8-b399-556e78e0ef42\") " Jul 07 13:20:44.668596 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.668536 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-util\") pod \"f457658d-bdeb-4d30-a224-6d995404d939\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " Jul 07 13:20:44.668596 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.668572 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzm68\" (UniqueName: \"kubernetes.io/projected/f457658d-bdeb-4d30-a224-6d995404d939-kube-api-access-pzm68\") pod \"f457658d-bdeb-4d30-a224-6d995404d939\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " Jul 07 13:20:44.668782 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.668598 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-bundle\") pod \"f457658d-bdeb-4d30-a224-6d995404d939\" (UID: \"f457658d-bdeb-4d30-a224-6d995404d939\") " Jul 07 13:20:44.669057 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.669026 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-bundle" (OuterVolumeSpecName: "bundle") pod "dde178da-32ab-4bc8-b399-556e78e0ef42" (UID: "dde178da-32ab-4bc8-b399-556e78e0ef42"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:20:44.669187 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.669116 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-bundle" (OuterVolumeSpecName: "bundle") pod "f457658d-bdeb-4d30-a224-6d995404d939" (UID: "f457658d-bdeb-4d30-a224-6d995404d939"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:20:44.671356 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.671325 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f457658d-bdeb-4d30-a224-6d995404d939-kube-api-access-pzm68" (OuterVolumeSpecName: "kube-api-access-pzm68") pod "f457658d-bdeb-4d30-a224-6d995404d939" (UID: "f457658d-bdeb-4d30-a224-6d995404d939"). InnerVolumeSpecName "kube-api-access-pzm68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:20:44.671464 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.671367 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde178da-32ab-4bc8-b399-556e78e0ef42-kube-api-access-xxrrz" (OuterVolumeSpecName: "kube-api-access-xxrrz") pod "dde178da-32ab-4bc8-b399-556e78e0ef42" (UID: "dde178da-32ab-4bc8-b399-556e78e0ef42"). InnerVolumeSpecName "kube-api-access-xxrrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:20:44.673381 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.673357 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-util" (OuterVolumeSpecName: "util") pod "f457658d-bdeb-4d30-a224-6d995404d939" (UID: "f457658d-bdeb-4d30-a224-6d995404d939"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:20:44.770037 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.769998 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-bundle\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:44.770037 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.770031 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxrrz\" (UniqueName: \"kubernetes.io/projected/dde178da-32ab-4bc8-b399-556e78e0ef42-kube-api-access-xxrrz\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:44.770037 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.770041 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-util\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:44.770312 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.770049 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzm68\" (UniqueName: \"kubernetes.io/projected/f457658d-bdeb-4d30-a224-6d995404d939-kube-api-access-pzm68\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:44.770312 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.770061 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f457658d-bdeb-4d30-a224-6d995404d939-bundle\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:44.782380 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.782346 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-util" (OuterVolumeSpecName: "util") pod "dde178da-32ab-4bc8-b399-556e78e0ef42" (UID: "dde178da-32ab-4bc8-b399-556e78e0ef42"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:20:44.871209 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:44.871173 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dde178da-32ab-4bc8-b399-556e78e0ef42-util\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:45.451425 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.451396 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" Jul 07 13:20:45.451425 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.451413 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx" event={"ID":"dde178da-32ab-4bc8-b399-556e78e0ef42","Type":"ContainerDied","Data":"687909a01192c4aa031ad54a19252b251a14b2fb1255884b3210c5eb789ad330"} Jul 07 13:20:45.451959 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.451464 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="687909a01192c4aa031ad54a19252b251a14b2fb1255884b3210c5eb789ad330" Jul 07 13:20:45.453632 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.453594 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" event={"ID":"f457658d-bdeb-4d30-a224-6d995404d939","Type":"ContainerDied","Data":"41fd432b596381dcb1001f49310db98519365368f09a8b69ef2adce6da897c41"} Jul 07 13:20:45.453752 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.453648 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41fd432b596381dcb1001f49310db98519365368f09a8b69ef2adce6da897c41" Jul 07 13:20:45.453823 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.453808 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb" Jul 07 13:20:45.587796 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.587762 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:45.678125 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.678085 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvs5v\" (UniqueName: \"kubernetes.io/projected/017bced3-b77f-407d-b471-00700fd3deac-kube-api-access-lvs5v\") pod \"017bced3-b77f-407d-b471-00700fd3deac\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " Jul 07 13:20:45.678309 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.678141 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-bundle\") pod \"017bced3-b77f-407d-b471-00700fd3deac\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " Jul 07 13:20:45.678309 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.678158 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-util\") pod \"017bced3-b77f-407d-b471-00700fd3deac\" (UID: \"017bced3-b77f-407d-b471-00700fd3deac\") " Jul 07 13:20:45.678804 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.678767 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-bundle" (OuterVolumeSpecName: "bundle") pod "017bced3-b77f-407d-b471-00700fd3deac" (UID: "017bced3-b77f-407d-b471-00700fd3deac"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:20:45.680636 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.680601 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/017bced3-b77f-407d-b471-00700fd3deac-kube-api-access-lvs5v" (OuterVolumeSpecName: "kube-api-access-lvs5v") pod "017bced3-b77f-407d-b471-00700fd3deac" (UID: "017bced3-b77f-407d-b471-00700fd3deac"). InnerVolumeSpecName "kube-api-access-lvs5v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:20:45.684290 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.684262 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-util" (OuterVolumeSpecName: "util") pod "017bced3-b77f-407d-b471-00700fd3deac" (UID: "017bced3-b77f-407d-b471-00700fd3deac"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:20:45.779725 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.779681 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvs5v\" (UniqueName: \"kubernetes.io/projected/017bced3-b77f-407d-b471-00700fd3deac-kube-api-access-lvs5v\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:45.779725 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.779716 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-bundle\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:45.779725 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:45.779728 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/017bced3-b77f-407d-b471-00700fd3deac-util\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:20:46.460702 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:46.460663 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" event={"ID":"017bced3-b77f-407d-b471-00700fd3deac","Type":"ContainerDied","Data":"e99b7d9af5de306e4ef161aea6d2faa07248638263f3f89169e5b4a73b5cfdad"} Jul 07 13:20:46.460702 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:46.460687 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64" Jul 07 13:20:46.461168 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:46.460716 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e99b7d9af5de306e4ef161aea6d2faa07248638263f3f89169e5b4a73b5cfdad" Jul 07 13:20:55.500003 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.499914 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc"] Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500283 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde178da-32ab-4bc8-b399-556e78e0ef42" containerName="extract" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500299 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde178da-32ab-4bc8-b399-556e78e0ef42" containerName="extract" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500310 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="017bced3-b77f-407d-b471-00700fd3deac" containerName="util" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500316 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="017bced3-b77f-407d-b471-00700fd3deac" containerName="util" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500324 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8500662d-2cd4-4a8c-a6ae-693736363282" containerName="util" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500329 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8500662d-2cd4-4a8c-a6ae-693736363282" containerName="util" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500338 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8500662d-2cd4-4a8c-a6ae-693736363282" containerName="extract" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500344 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8500662d-2cd4-4a8c-a6ae-693736363282" containerName="extract" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500350 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="017bced3-b77f-407d-b471-00700fd3deac" containerName="extract" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500355 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="017bced3-b77f-407d-b471-00700fd3deac" containerName="extract" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500362 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f457658d-bdeb-4d30-a224-6d995404d939" containerName="extract" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500366 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f457658d-bdeb-4d30-a224-6d995404d939" containerName="extract" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500375 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde178da-32ab-4bc8-b399-556e78e0ef42" containerName="util" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500381 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde178da-32ab-4bc8-b399-556e78e0ef42" containerName="util" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500387 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f457658d-bdeb-4d30-a224-6d995404d939" containerName="util" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500392 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f457658d-bdeb-4d30-a224-6d995404d939" containerName="util" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500399 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f457658d-bdeb-4d30-a224-6d995404d939" containerName="pull" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500404 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f457658d-bdeb-4d30-a224-6d995404d939" containerName="pull" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500412 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde178da-32ab-4bc8-b399-556e78e0ef42" containerName="pull" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500418 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde178da-32ab-4bc8-b399-556e78e0ef42" containerName="pull" Jul 07 13:20:55.500411 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500423 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="017bced3-b77f-407d-b471-00700fd3deac" containerName="pull" Jul 07 13:20:55.501011 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500428 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="017bced3-b77f-407d-b471-00700fd3deac" containerName="pull" Jul 07 13:20:55.501011 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500435 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8500662d-2cd4-4a8c-a6ae-693736363282" containerName="pull" Jul 07 13:20:55.501011 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500441 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8500662d-2cd4-4a8c-a6ae-693736363282" containerName="pull" Jul 07 13:20:55.501011 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500499 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8500662d-2cd4-4a8c-a6ae-693736363282" containerName="extract" Jul 07 13:20:55.501011 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500508 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde178da-32ab-4bc8-b399-556e78e0ef42" containerName="extract" Jul 07 13:20:55.501011 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500515 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f457658d-bdeb-4d30-a224-6d995404d939" containerName="extract" Jul 07 13:20:55.501011 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.500521 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="017bced3-b77f-407d-b471-00700fd3deac" containerName="extract" Jul 07 13:20:55.502226 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.502209 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc" Jul 07 13:20:55.505208 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.505184 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hbgfk\"" Jul 07 13:20:55.536192 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.536154 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc"] Jul 07 13:20:55.560230 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.560190 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb2wx\" (UniqueName: \"kubernetes.io/projected/45159cff-f6eb-4e25-84ae-6a3999fa501c-kube-api-access-sb2wx\") pod \"authorino-operator-75cc8d5cb7-mnxbc\" (UID: \"45159cff-f6eb-4e25-84ae-6a3999fa501c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc" Jul 07 13:20:55.661201 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.661169 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb2wx\" (UniqueName: \"kubernetes.io/projected/45159cff-f6eb-4e25-84ae-6a3999fa501c-kube-api-access-sb2wx\") pod \"authorino-operator-75cc8d5cb7-mnxbc\" (UID: \"45159cff-f6eb-4e25-84ae-6a3999fa501c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc" Jul 07 13:20:55.673910 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.673871 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb2wx\" (UniqueName: \"kubernetes.io/projected/45159cff-f6eb-4e25-84ae-6a3999fa501c-kube-api-access-sb2wx\") pod \"authorino-operator-75cc8d5cb7-mnxbc\" (UID: \"45159cff-f6eb-4e25-84ae-6a3999fa501c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc" Jul 07 13:20:55.812182 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.812092 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc" Jul 07 13:20:55.966828 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:55.966801 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc"] Jul 07 13:20:55.968981 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:20:55.968948 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45159cff_f6eb_4e25_84ae_6a3999fa501c.slice/crio-b93de125f00c26776a3f67e042a308406dc8999b053096082f3115c91e455c22 WatchSource:0}: Error finding container b93de125f00c26776a3f67e042a308406dc8999b053096082f3115c91e455c22: Status 404 returned error can't find the container with id b93de125f00c26776a3f67e042a308406dc8999b053096082f3115c91e455c22 Jul 07 13:20:56.515661 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:56.515626 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc" event={"ID":"45159cff-f6eb-4e25-84ae-6a3999fa501c","Type":"ContainerStarted","Data":"b93de125f00c26776a3f67e042a308406dc8999b053096082f3115c91e455c22"} Jul 07 13:20:58.529207 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:58.529162 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc" event={"ID":"45159cff-f6eb-4e25-84ae-6a3999fa501c","Type":"ContainerStarted","Data":"198c3a498776718d930bce861951393d306c376b42f836d2619036fa3b1e1083"} Jul 07 13:20:58.529652 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:58.529322 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc" Jul 07 13:20:58.556119 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:20:58.555996 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc" podStartSLOduration=1.242944081 podStartE2EDuration="3.555973718s" totalImagesPullingTime="2.313029637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:20:58.554521382 +0000 UTC m=+453.608943061" watchObservedRunningTime="2026-07-07 13:20:58.555973718 +0000 UTC m=+453.610395375" Jul 07 13:21:09.178359 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.178323 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q"] Jul 07 13:21:09.185933 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.185903 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:09.189333 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.189309 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mszc8\"" Jul 07 13:21:09.224394 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.224355 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q"] Jul 07 13:21:09.282091 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.282044 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/703fe70a-b4d0-4ccf-99a1-4d21d823accb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5m55q\" (UID: \"703fe70a-b4d0-4ccf-99a1-4d21d823accb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:09.282311 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.282192 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w4vt\" (UniqueName: \"kubernetes.io/projected/703fe70a-b4d0-4ccf-99a1-4d21d823accb-kube-api-access-8w4vt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5m55q\" (UID: \"703fe70a-b4d0-4ccf-99a1-4d21d823accb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:09.383199 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.383155 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w4vt\" (UniqueName: \"kubernetes.io/projected/703fe70a-b4d0-4ccf-99a1-4d21d823accb-kube-api-access-8w4vt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5m55q\" (UID: \"703fe70a-b4d0-4ccf-99a1-4d21d823accb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:09.383199 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.383205 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/703fe70a-b4d0-4ccf-99a1-4d21d823accb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5m55q\" (UID: \"703fe70a-b4d0-4ccf-99a1-4d21d823accb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:09.383570 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.383553 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/703fe70a-b4d0-4ccf-99a1-4d21d823accb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5m55q\" (UID: \"703fe70a-b4d0-4ccf-99a1-4d21d823accb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:09.396971 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.396933 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w4vt\" (UniqueName: \"kubernetes.io/projected/703fe70a-b4d0-4ccf-99a1-4d21d823accb-kube-api-access-8w4vt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5m55q\" (UID: \"703fe70a-b4d0-4ccf-99a1-4d21d823accb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:09.496954 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.496906 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:09.537136 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.537108 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mnxbc" Jul 07 13:21:09.665414 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:09.665379 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q"] Jul 07 13:21:09.669127 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:21:09.669091 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703fe70a_b4d0_4ccf_99a1_4d21d823accb.slice/crio-b09d3a27fcccee550e80a71dbb2e9b648073e79144f3763bdf22803660ec8016 WatchSource:0}: Error finding container b09d3a27fcccee550e80a71dbb2e9b648073e79144f3763bdf22803660ec8016: Status 404 returned error can't find the container with id b09d3a27fcccee550e80a71dbb2e9b648073e79144f3763bdf22803660ec8016 Jul 07 13:21:10.595503 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:10.595467 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" event={"ID":"703fe70a-b4d0-4ccf-99a1-4d21d823accb","Type":"ContainerStarted","Data":"b09d3a27fcccee550e80a71dbb2e9b648073e79144f3763bdf22803660ec8016"} Jul 07 13:21:17.641281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:17.641224 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" event={"ID":"703fe70a-b4d0-4ccf-99a1-4d21d823accb","Type":"ContainerStarted","Data":"4323efb4222812d284a0f9eeef381cb7a50c00676c056d6be46512870a830cff"} Jul 07 13:21:17.641851 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:17.641476 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:17.698458 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:17.698395 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" podStartSLOduration=1.277284641 podStartE2EDuration="8.698378209s" totalImagesPullingTime="7.421093568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:21:17.6976123 +0000 UTC m=+472.752033959" watchObservedRunningTime="2026-07-07 13:21:17.698378209 +0000 UTC m=+472.752799866" Jul 07 13:21:28.648951 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:28.648915 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:30.210781 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.210722 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2"] Jul 07 13:21:30.219360 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.219322 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:30.229341 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.229311 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2"] Jul 07 13:21:30.270076 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.270027 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mslv2\" (UID: \"5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:30.270076 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.270073 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdr2q\" (UniqueName: \"kubernetes.io/projected/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-kube-api-access-kdr2q\") pod \"kuadrant-operator-controller-manager-55467d9f7-mslv2\" (UID: \"5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:30.370577 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.370531 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mslv2\" (UID: \"5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:30.370577 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.370576 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdr2q\" (UniqueName: \"kubernetes.io/projected/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-kube-api-access-kdr2q\") pod \"kuadrant-operator-controller-manager-55467d9f7-mslv2\" (UID: \"5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:30.370966 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.370941 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-mslv2\" (UID: \"5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:30.379944 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.379906 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdr2q\" (UniqueName: \"kubernetes.io/projected/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-kube-api-access-kdr2q\") pod \"kuadrant-operator-controller-manager-55467d9f7-mslv2\" (UID: \"5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:30.530856 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.530752 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:30.571415 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.571373 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2"] Jul 07 13:21:30.579075 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.579040 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2"] Jul 07 13:21:30.587614 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.587569 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q"] Jul 07 13:21:30.587868 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.587840 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" podUID="703fe70a-b4d0-4ccf-99a1-4d21d823accb" containerName="manager" containerID="cri-o://4323efb4222812d284a0f9eeef381cb7a50c00676c056d6be46512870a830cff" gracePeriod=2 Jul 07 13:21:30.604441 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.604404 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q"] Jul 07 13:21:30.607745 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:21:30.607693 2574 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"703fe70a-b4d0-4ccf-99a1-4d21d823accb\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-5m55q\": pods \"kuadrant-operator-controller-manager-8684c9ff49-5m55q\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:30.627038 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.626999 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd"] Jul 07 13:21:30.627437 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.627418 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="703fe70a-b4d0-4ccf-99a1-4d21d823accb" containerName="manager" Jul 07 13:21:30.627531 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.627439 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="703fe70a-b4d0-4ccf-99a1-4d21d823accb" containerName="manager" Jul 07 13:21:30.627583 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.627551 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="703fe70a-b4d0-4ccf-99a1-4d21d823accb" containerName="manager" Jul 07 13:21:30.630954 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.630924 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:21:30.637304 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:21:30.637260 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5m55q\" is forbidden: User \"system:node:ip-10-0-143-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-95.ec2.internal' and this object" podUID="703fe70a-b4d0-4ccf-99a1-4d21d823accb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:30.660203 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.660162 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd"] Jul 07 13:21:30.673615 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.673553 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drnb4\" (UniqueName: \"kubernetes.io/projected/a24059b2-bacd-405b-add6-d33937dca8c6-kube-api-access-drnb4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbphd\" (UID: \"a24059b2-bacd-405b-add6-d33937dca8c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:21:30.673781 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.673703 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a24059b2-bacd-405b-add6-d33937dca8c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbphd\" (UID: \"a24059b2-bacd-405b-add6-d33937dca8c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:21:30.689791 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.689750 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j"] Jul 07 13:21:30.693571 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.693545 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j" Jul 07 13:21:30.697307 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.697281 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-s9znd\"" Jul 07 13:21:30.706506 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.706471 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j"] Jul 07 13:21:30.717720 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.717675 2574 generic.go:356] "Generic (PLEG): container finished" podID="703fe70a-b4d0-4ccf-99a1-4d21d823accb" containerID="4323efb4222812d284a0f9eeef381cb7a50c00676c056d6be46512870a830cff" exitCode=0 Jul 07 13:21:30.721477 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:21:30.721440 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5m55q\" is forbidden: User \"system:node:ip-10-0-143-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-95.ec2.internal' and this object" podUID="703fe70a-b4d0-4ccf-99a1-4d21d823accb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:30.775055 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.775019 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a24059b2-bacd-405b-add6-d33937dca8c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbphd\" (UID: \"a24059b2-bacd-405b-add6-d33937dca8c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:21:30.775743 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.775091 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drnb4\" (UniqueName: \"kubernetes.io/projected/a24059b2-bacd-405b-add6-d33937dca8c6-kube-api-access-drnb4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbphd\" (UID: \"a24059b2-bacd-405b-add6-d33937dca8c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:21:30.775743 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.775209 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gvvh\" (UniqueName: \"kubernetes.io/projected/8199209b-a00e-410e-9e8d-d6e2ac6230b3-kube-api-access-5gvvh\") pod \"limitador-operator-controller-manager-859d7b78bd-tqf8j\" (UID: \"8199209b-a00e-410e-9e8d-d6e2ac6230b3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j" Jul 07 13:21:30.775743 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.775647 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a24059b2-bacd-405b-add6-d33937dca8c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbphd\" (UID: \"a24059b2-bacd-405b-add6-d33937dca8c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:21:30.785020 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.784946 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drnb4\" (UniqueName: \"kubernetes.io/projected/a24059b2-bacd-405b-add6-d33937dca8c6-kube-api-access-drnb4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pbphd\" (UID: \"a24059b2-bacd-405b-add6-d33937dca8c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:21:30.834399 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.834369 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:30.836646 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:21:30.836614 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5m55q\" is forbidden: User \"system:node:ip-10-0-143-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-95.ec2.internal' and this object" podUID="703fe70a-b4d0-4ccf-99a1-4d21d823accb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:30.875796 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.875758 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/703fe70a-b4d0-4ccf-99a1-4d21d823accb-extensions-socket-volume\") pod \"703fe70a-b4d0-4ccf-99a1-4d21d823accb\" (UID: \"703fe70a-b4d0-4ccf-99a1-4d21d823accb\") " Jul 07 13:21:30.875796 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.875799 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8w4vt\" (UniqueName: \"kubernetes.io/projected/703fe70a-b4d0-4ccf-99a1-4d21d823accb-kube-api-access-8w4vt\") pod \"703fe70a-b4d0-4ccf-99a1-4d21d823accb\" (UID: \"703fe70a-b4d0-4ccf-99a1-4d21d823accb\") " Jul 07 13:21:30.876016 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.875948 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gvvh\" (UniqueName: \"kubernetes.io/projected/8199209b-a00e-410e-9e8d-d6e2ac6230b3-kube-api-access-5gvvh\") pod \"limitador-operator-controller-manager-859d7b78bd-tqf8j\" (UID: \"8199209b-a00e-410e-9e8d-d6e2ac6230b3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j" Jul 07 13:21:30.876181 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.876153 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/703fe70a-b4d0-4ccf-99a1-4d21d823accb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "703fe70a-b4d0-4ccf-99a1-4d21d823accb" (UID: "703fe70a-b4d0-4ccf-99a1-4d21d823accb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:21:30.878141 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.878115 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/703fe70a-b4d0-4ccf-99a1-4d21d823accb-kube-api-access-8w4vt" (OuterVolumeSpecName: "kube-api-access-8w4vt") pod "703fe70a-b4d0-4ccf-99a1-4d21d823accb" (UID: "703fe70a-b4d0-4ccf-99a1-4d21d823accb"). InnerVolumeSpecName "kube-api-access-8w4vt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:21:30.886666 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.886642 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gvvh\" (UniqueName: \"kubernetes.io/projected/8199209b-a00e-410e-9e8d-d6e2ac6230b3-kube-api-access-5gvvh\") pod \"limitador-operator-controller-manager-859d7b78bd-tqf8j\" (UID: \"8199209b-a00e-410e-9e8d-d6e2ac6230b3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j" Jul 07 13:21:30.976909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.976866 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/703fe70a-b4d0-4ccf-99a1-4d21d823accb-extensions-socket-volume\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:21:30.976909 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.976901 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w4vt\" (UniqueName: \"kubernetes.io/projected/703fe70a-b4d0-4ccf-99a1-4d21d823accb-kube-api-access-8w4vt\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:21:30.977768 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:30.977742 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:21:31.005756 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.005720 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j" Jul 07 13:21:31.192148 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.192038 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd"] Jul 07 13:21:31.194270 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.194231 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j"] Jul 07 13:21:31.195936 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:21:31.195894 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda24059b2_bacd_405b_add6_d33937dca8c6.slice/crio-df20c2b735c97e13d4084f03b6e47e8bdff0ef1ea78acf7dc93df1ab1cec191d WatchSource:0}: Error finding container df20c2b735c97e13d4084f03b6e47e8bdff0ef1ea78acf7dc93df1ab1cec191d: Status 404 returned error can't find the container with id df20c2b735c97e13d4084f03b6e47e8bdff0ef1ea78acf7dc93df1ab1cec191d Jul 07 13:21:31.196719 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:21:31.196690 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8199209b_a00e_410e_9e8d_d6e2ac6230b3.slice/crio-939790a600129ffae10a6b1c0253240509255ccc576014784bf11b20c791b8ed WatchSource:0}: Error finding container 939790a600129ffae10a6b1c0253240509255ccc576014784bf11b20c791b8ed: Status 404 returned error can't find the container with id 939790a600129ffae10a6b1c0253240509255ccc576014784bf11b20c791b8ed Jul 07 13:21:31.433764 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.433718 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="703fe70a-b4d0-4ccf-99a1-4d21d823accb" path="/var/lib/kubelet/pods/703fe70a-b4d0-4ccf-99a1-4d21d823accb/volumes" Jul 07 13:21:31.541077 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:21:31.541039 2574 log.go:32] "RunPodSandbox from runtime service failed" err=< Jul 07 13:21:31.541077 ip-10-0-143-95 kubenswrapper[2574]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-mslv2_kuadrant-system_5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048_0(dbf244ced5cacff0101cf1b013285ad8bf5307086b7621fa6d33c4797fbc1141): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-mslv2 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"dbf244ced5cacff0101cf1b013285ad8bf5307086b7621fa6d33c4797fbc1141" Netns:"/var/run/netns/cc4513e9-6dcf-410c-9e62-74cffa546731" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-mslv2;K8S_POD_INFRA_CONTAINER_ID=dbf244ced5cacff0101cf1b013285ad8bf5307086b7621fa6d33c4797fbc1141;K8S_POD_UID=5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2 dbf244ced5cacff0101cf1b013285ad8bf5307086b7621fa6d33c4797fbc1141 network default NAD default NAD key ] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2 dbf244ced5cacff0101cf1b013285ad8bf5307086b7621fa6d33c4797fbc1141 network default NAD default NAD key ] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-55467d9f7-mslv2" not found Jul 07 13:21:31.541077 ip-10-0-143-95 kubenswrapper[2574]: ' Jul 07 13:21:31.541077 ip-10-0-143-95 kubenswrapper[2574]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 07 13:21:31.541077 ip-10-0-143-95 kubenswrapper[2574]: > Jul 07 13:21:31.541323 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:21:31.541111 2574 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=< Jul 07 13:21:31.541323 ip-10-0-143-95 kubenswrapper[2574]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-mslv2_kuadrant-system_5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048_0(dbf244ced5cacff0101cf1b013285ad8bf5307086b7621fa6d33c4797fbc1141): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-mslv2 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"dbf244ced5cacff0101cf1b013285ad8bf5307086b7621fa6d33c4797fbc1141" Netns:"/var/run/netns/cc4513e9-6dcf-410c-9e62-74cffa546731" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-mslv2;K8S_POD_INFRA_CONTAINER_ID=dbf244ced5cacff0101cf1b013285ad8bf5307086b7621fa6d33c4797fbc1141;K8S_POD_UID=5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2 dbf244ced5cacff0101cf1b013285ad8bf5307086b7621fa6d33c4797fbc1141 network default NAD default NAD key ] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2 dbf244ced5cacff0101cf1b013285ad8bf5307086b7621fa6d33c4797fbc1141 network default NAD default NAD key ] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-55467d9f7-mslv2" not found Jul 07 13:21:31.541323 ip-10-0-143-95 kubenswrapper[2574]: ' Jul 07 13:21:31.541323 ip-10-0-143-95 kubenswrapper[2574]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 07 13:21:31.541323 ip-10-0-143-95 kubenswrapper[2574]: > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:31.726267 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.726216 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" event={"ID":"a24059b2-bacd-405b-add6-d33937dca8c6","Type":"ContainerStarted","Data":"605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11"} Jul 07 13:21:31.726480 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.726345 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" event={"ID":"a24059b2-bacd-405b-add6-d33937dca8c6","Type":"ContainerStarted","Data":"df20c2b735c97e13d4084f03b6e47e8bdff0ef1ea78acf7dc93df1ab1cec191d"} Jul 07 13:21:31.726480 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.726384 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:21:31.727908 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.727840 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5m55q" Jul 07 13:21:31.728073 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.727845 2574 scope.go:117] "RemoveContainer" containerID="4323efb4222812d284a0f9eeef381cb7a50c00676c056d6be46512870a830cff" Jul 07 13:21:31.729650 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.729602 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j" event={"ID":"8199209b-a00e-410e-9e8d-d6e2ac6230b3","Type":"ContainerStarted","Data":"939790a600129ffae10a6b1c0253240509255ccc576014784bf11b20c791b8ed"} Jul 07 13:21:31.729802 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.729706 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:31.737565 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.737536 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:31.761990 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.761930 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" podStartSLOduration=1.761910184 podStartE2EDuration="1.761910184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:21:31.760428604 +0000 UTC m=+486.814850300" watchObservedRunningTime="2026-07-07 13:21:31.761910184 +0000 UTC m=+486.816331843" Jul 07 13:21:31.763067 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:21:31.763026 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mslv2\" is forbidden: User \"system:node:ip-10-0-143-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-95.ec2.internal' and this object" podUID="5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:31.784141 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.784102 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-extensions-socket-volume\") pod \"5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048\" (UID: \"5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048\") " Jul 07 13:21:31.784347 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.784199 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdr2q\" (UniqueName: \"kubernetes.io/projected/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-kube-api-access-kdr2q\") pod \"5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048\" (UID: \"5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048\") " Jul 07 13:21:31.784463 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.784430 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048" (UID: "5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:21:31.784593 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.784535 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-extensions-socket-volume\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:21:31.789575 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.789523 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-kube-api-access-kdr2q" (OuterVolumeSpecName: "kube-api-access-kdr2q") pod "5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048" (UID: "5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048"). InnerVolumeSpecName "kube-api-access-kdr2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:21:31.887347 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:31.885662 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdr2q\" (UniqueName: \"kubernetes.io/projected/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048-kube-api-access-kdr2q\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:21:32.735406 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:32.735373 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:32.740893 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:21:32.740858 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mslv2\" is forbidden: User \"system:node:ip-10-0-143-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-95.ec2.internal' and this object" podUID="5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:32.744153 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:21:32.744121 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-mslv2\" is forbidden: User \"system:node:ip-10-0-143-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-95.ec2.internal' and this object" podUID="5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-mslv2" Jul 07 13:21:33.433072 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:33.433032 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048" path="/var/lib/kubelet/pods/5dc75a9a-74ab-4dfe-a9ea-8b0c7d03a048/volumes" Jul 07 13:21:33.745792 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:33.745752 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j" event={"ID":"8199209b-a00e-410e-9e8d-d6e2ac6230b3","Type":"ContainerStarted","Data":"911de4878e39248b4bf5129f463fac2dafefd5406b7efee4421c8fa897063397"} Jul 07 13:21:33.746175 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:33.745836 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j" Jul 07 13:21:33.776016 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:33.775953 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j" podStartSLOduration=1.7039996309999998 podStartE2EDuration="3.775931727s" totalImagesPullingTime="2.071932096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:21:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:21:33.774474654 +0000 UTC m=+488.828896327" watchObservedRunningTime="2026-07-07 13:21:33.775931727 +0000 UTC m=+488.830353382" Jul 07 13:21:42.738544 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:42.738507 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:21:44.753676 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:44.753640 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqf8j" Jul 07 13:21:55.228373 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:21:55.228333 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db558966b-kpjg4"] Jul 07 13:22:01.224852 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.224814 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd"] Jul 07 13:22:01.225472 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.225027 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" podUID="a24059b2-bacd-405b-add6-d33937dca8c6" containerName="manager" containerID="cri-o://605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11" gracePeriod=10 Jul 07 13:22:01.486159 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.486076 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:22:01.544107 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.544076 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a24059b2-bacd-405b-add6-d33937dca8c6-extensions-socket-volume\") pod \"a24059b2-bacd-405b-add6-d33937dca8c6\" (UID: \"a24059b2-bacd-405b-add6-d33937dca8c6\") " Jul 07 13:22:01.544311 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.544126 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drnb4\" (UniqueName: \"kubernetes.io/projected/a24059b2-bacd-405b-add6-d33937dca8c6-kube-api-access-drnb4\") pod \"a24059b2-bacd-405b-add6-d33937dca8c6\" (UID: \"a24059b2-bacd-405b-add6-d33937dca8c6\") " Jul 07 13:22:01.544534 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.544503 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a24059b2-bacd-405b-add6-d33937dca8c6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a24059b2-bacd-405b-add6-d33937dca8c6" (UID: "a24059b2-bacd-405b-add6-d33937dca8c6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:01.546436 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.546408 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a24059b2-bacd-405b-add6-d33937dca8c6-kube-api-access-drnb4" (OuterVolumeSpecName: "kube-api-access-drnb4") pod "a24059b2-bacd-405b-add6-d33937dca8c6" (UID: "a24059b2-bacd-405b-add6-d33937dca8c6"). InnerVolumeSpecName "kube-api-access-drnb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:22:01.596624 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.596581 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4"] Jul 07 13:22:01.596966 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.596951 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a24059b2-bacd-405b-add6-d33937dca8c6" containerName="manager" Jul 07 13:22:01.597032 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.596967 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24059b2-bacd-405b-add6-d33937dca8c6" containerName="manager" Jul 07 13:22:01.597068 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.597050 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a24059b2-bacd-405b-add6-d33937dca8c6" containerName="manager" Jul 07 13:22:01.600111 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.600092 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:22:01.634778 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.634742 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4"] Jul 07 13:22:01.645854 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.645813 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a24059b2-bacd-405b-add6-d33937dca8c6-extensions-socket-volume\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:22:01.645854 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.645848 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drnb4\" (UniqueName: \"kubernetes.io/projected/a24059b2-bacd-405b-add6-d33937dca8c6-kube-api-access-drnb4\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:22:01.747319 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.747168 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfb6t\" (UniqueName: \"kubernetes.io/projected/ca6747bf-09a6-4990-b1e8-ce15531497cd-kube-api-access-pfb6t\") pod \"kuadrant-operator-controller-manager-86878c769f-pk8s4\" (UID: \"ca6747bf-09a6-4990-b1e8-ce15531497cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:22:01.747476 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.747346 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca6747bf-09a6-4990-b1e8-ce15531497cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pk8s4\" (UID: \"ca6747bf-09a6-4990-b1e8-ce15531497cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:22:01.848179 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.848134 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca6747bf-09a6-4990-b1e8-ce15531497cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pk8s4\" (UID: \"ca6747bf-09a6-4990-b1e8-ce15531497cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:22:01.848179 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.848186 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfb6t\" (UniqueName: \"kubernetes.io/projected/ca6747bf-09a6-4990-b1e8-ce15531497cd-kube-api-access-pfb6t\") pod \"kuadrant-operator-controller-manager-86878c769f-pk8s4\" (UID: \"ca6747bf-09a6-4990-b1e8-ce15531497cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:22:01.848570 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.848550 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca6747bf-09a6-4990-b1e8-ce15531497cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pk8s4\" (UID: \"ca6747bf-09a6-4990-b1e8-ce15531497cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:22:01.857484 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.857460 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfb6t\" (UniqueName: \"kubernetes.io/projected/ca6747bf-09a6-4990-b1e8-ce15531497cd-kube-api-access-pfb6t\") pod \"kuadrant-operator-controller-manager-86878c769f-pk8s4\" (UID: \"ca6747bf-09a6-4990-b1e8-ce15531497cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:22:01.894502 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.894465 2574 generic.go:356] "Generic (PLEG): container finished" podID="a24059b2-bacd-405b-add6-d33937dca8c6" containerID="605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11" exitCode=0 Jul 07 13:22:01.894684 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.894509 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" Jul 07 13:22:01.894684 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.894539 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" event={"ID":"a24059b2-bacd-405b-add6-d33937dca8c6","Type":"ContainerDied","Data":"605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11"} Jul 07 13:22:01.894684 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.894592 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd" event={"ID":"a24059b2-bacd-405b-add6-d33937dca8c6","Type":"ContainerDied","Data":"df20c2b735c97e13d4084f03b6e47e8bdff0ef1ea78acf7dc93df1ab1cec191d"} Jul 07 13:22:01.894684 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.894615 2574 scope.go:117] "RemoveContainer" containerID="605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11" Jul 07 13:22:01.905057 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.905036 2574 scope.go:117] "RemoveContainer" containerID="605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11" Jul 07 13:22:01.905415 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:22:01.905387 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11\": container with ID starting with 605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11 not found: ID does not exist" containerID="605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11" Jul 07 13:22:01.905483 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.905419 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11"} err="failed to get container status \"605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11\": rpc error: code = NotFound desc = could not find container \"605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11\": container with ID starting with 605f712b7ddee401d8c1b0a1e281c5f16bcb0433a9fa7f3946b81ac8325bba11 not found: ID does not exist" Jul 07 13:22:01.909717 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.909695 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:22:01.932335 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.932296 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd"] Jul 07 13:22:01.941164 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:01.941127 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pbphd"] Jul 07 13:22:02.076459 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:02.076412 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4"] Jul 07 13:22:02.082406 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:22:02.082377 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6747bf_09a6_4990_b1e8_ce15531497cd.slice/crio-54fe1c45139246126346b3e5d6a165765f83e8e0e82571cb827ebdbb89717413 WatchSource:0}: Error finding container 54fe1c45139246126346b3e5d6a165765f83e8e0e82571cb827ebdbb89717413: Status 404 returned error can't find the container with id 54fe1c45139246126346b3e5d6a165765f83e8e0e82571cb827ebdbb89717413 Jul 07 13:22:02.902014 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:02.901972 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" event={"ID":"ca6747bf-09a6-4990-b1e8-ce15531497cd","Type":"ContainerStarted","Data":"819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4"} Jul 07 13:22:02.902520 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:02.902037 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" event={"ID":"ca6747bf-09a6-4990-b1e8-ce15531497cd","Type":"ContainerStarted","Data":"54fe1c45139246126346b3e5d6a165765f83e8e0e82571cb827ebdbb89717413"} Jul 07 13:22:02.902520 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:02.902065 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:22:02.967710 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:02.967631 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" podStartSLOduration=1.967613536 podStartE2EDuration="1.967613536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:02.960183775 +0000 UTC m=+518.014605432" watchObservedRunningTime="2026-07-07 13:22:02.967613536 +0000 UTC m=+518.022035194" Jul 07 13:22:03.433132 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:03.433091 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a24059b2-bacd-405b-add6-d33937dca8c6" path="/var/lib/kubelet/pods/a24059b2-bacd-405b-add6-d33937dca8c6/volumes" Jul 07 13:22:13.911786 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:13.911749 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:22:20.249080 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.248985 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5db558966b-kpjg4" podUID="4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" containerName="console" containerID="cri-o://b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5" gracePeriod=15 Jul 07 13:22:20.518894 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.518865 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:22:20.612319 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.612275 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xppxm\" (UniqueName: \"kubernetes.io/projected/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-kube-api-access-xppxm\") pod \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " Jul 07 13:22:20.612319 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.612326 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-config\") pod \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " Jul 07 13:22:20.612575 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.612357 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-trusted-ca-bundle\") pod \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " Jul 07 13:22:20.612575 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.612411 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-service-ca\") pod \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " Jul 07 13:22:20.612575 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.612447 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-serving-cert\") pod \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " Jul 07 13:22:20.612575 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.612487 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-oauth-config\") pod \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " Jul 07 13:22:20.612575 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.612515 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-oauth-serving-cert\") pod \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\" (UID: \"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b\") " Jul 07 13:22:20.612837 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.612799 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-config" (OuterVolumeSpecName: "console-config") pod "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" (UID: "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:22:20.612905 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.612842 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" (UID: "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:22:20.612905 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.612852 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-service-ca" (OuterVolumeSpecName: "service-ca") pod "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" (UID: "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:22:20.613132 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.613098 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" (UID: "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:22:20.614644 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.614620 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-kube-api-access-xppxm" (OuterVolumeSpecName: "kube-api-access-xppxm") pod "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" (UID: "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b"). InnerVolumeSpecName "kube-api-access-xppxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:22:20.614644 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.614643 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" (UID: "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:22:20.614844 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.614727 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" (UID: "4dd09eb8-d752-4a3a-ac75-25f4ee2da02b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:22:20.714158 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.714121 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xppxm\" (UniqueName: \"kubernetes.io/projected/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-kube-api-access-xppxm\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:22:20.714158 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.714153 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-config\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:22:20.714158 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.714166 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-trusted-ca-bundle\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:22:20.714469 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.714175 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:22:20.714469 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.714184 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-serving-cert\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:22:20.714469 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.714192 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-console-oauth-config\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:22:20.714469 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:20.714200 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b-oauth-serving-cert\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:22:21.004771 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:21.004729 2574 generic.go:356] "Generic (PLEG): container finished" podID="4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" containerID="b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5" exitCode=0 Jul 07 13:22:21.004771 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:21.004775 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db558966b-kpjg4" Jul 07 13:22:21.005019 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:21.004808 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db558966b-kpjg4" event={"ID":"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b","Type":"ContainerDied","Data":"b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5"} Jul 07 13:22:21.005019 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:21.004858 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db558966b-kpjg4" event={"ID":"4dd09eb8-d752-4a3a-ac75-25f4ee2da02b","Type":"ContainerDied","Data":"6e6147f05e4bd9235cc62e72adaeb55d9721ec2ffc31ee8e0c7f2b863ebb956a"} Jul 07 13:22:21.005019 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:21.004881 2574 scope.go:117] "RemoveContainer" containerID="b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5" Jul 07 13:22:21.014745 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:21.014724 2574 scope.go:117] "RemoveContainer" containerID="b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5" Jul 07 13:22:21.015071 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:22:21.015050 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5\": container with ID starting with b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5 not found: ID does not exist" containerID="b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5" Jul 07 13:22:21.015136 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:21.015083 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5"} err="failed to get container status \"b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5\": rpc error: code = NotFound desc = could not find container \"b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5\": container with ID starting with b659f834d89b657b818a1e9d5e3b75d59d6c9f6407c58cae8f5ff6de16f4fec5 not found: ID does not exist" Jul 07 13:22:21.036475 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:21.036430 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db558966b-kpjg4"] Jul 07 13:22:21.051360 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:21.051326 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5db558966b-kpjg4"] Jul 07 13:22:21.432901 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:22:21.432812 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" path="/var/lib/kubelet/pods/4dd09eb8-d752-4a3a-ac75-25f4ee2da02b/volumes" Jul 07 13:24:01.309809 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.309720 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb"] Jul 07 13:24:01.312306 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.310109 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" containerName="console" Jul 07 13:24:01.312306 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.310123 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" containerName="console" Jul 07 13:24:01.312306 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.310198 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dd09eb8-d752-4a3a-ac75-25f4ee2da02b" containerName="console" Jul 07 13:24:01.313206 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.313188 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.316829 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.316804 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jn6x5\"" Jul 07 13:24:01.318427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.318406 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 13:24:01.318571 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.318503 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 13:24:01.319631 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.319614 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 13:24:01.333309 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.333271 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb"] Jul 07 13:24:01.372223 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.372189 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.372223 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.372235 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.372486 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.372281 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.372486 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.372358 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.372486 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.372412 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.372486 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.372444 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4b98\" (UniqueName: \"kubernetes.io/projected/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-kube-api-access-p4b98\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.473269 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.473210 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.473475 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.473299 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.473475 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.473331 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4b98\" (UniqueName: \"kubernetes.io/projected/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-kube-api-access-p4b98\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.473475 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.473382 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.473475 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.473424 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.473475 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.473448 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.473894 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.473852 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.474010 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.473891 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.474010 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.473932 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.475819 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.475797 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.476004 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.475989 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.482666 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.482643 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4b98\" (UniqueName: \"kubernetes.io/projected/d93924ae-d3cd-4c0b-97e6-263b0eeddc47-kube-api-access-p4b98\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pvdqb\" (UID: \"d93924ae-d3cd-4c0b-97e6-263b0eeddc47\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.623986 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.623889 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:01.767184 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.767155 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb"] Jul 07 13:24:01.768978 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:24:01.768946 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd93924ae_d3cd_4c0b_97e6_263b0eeddc47.slice/crio-1541cbf7e53cf3645a44b3c54da8a2046d9c6d0d93475d8a5f00ad3cbc02b794 WatchSource:0}: Error finding container 1541cbf7e53cf3645a44b3c54da8a2046d9c6d0d93475d8a5f00ad3cbc02b794: Status 404 returned error can't find the container with id 1541cbf7e53cf3645a44b3c54da8a2046d9c6d0d93475d8a5f00ad3cbc02b794 Jul 07 13:24:01.770540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:01.770519 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:24:02.520839 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:02.520787 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" event={"ID":"d93924ae-d3cd-4c0b-97e6-263b0eeddc47","Type":"ContainerStarted","Data":"1541cbf7e53cf3645a44b3c54da8a2046d9c6d0d93475d8a5f00ad3cbc02b794"} Jul 07 13:24:03.527328 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:03.527288 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" event={"ID":"d93924ae-d3cd-4c0b-97e6-263b0eeddc47","Type":"ContainerStarted","Data":"46a53289bb8b93c550a6e163f7b3497437d38cb9e9340dfbb64b27c9c0503fea"} Jul 07 13:24:03.552254 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:03.552202 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" podStartSLOduration=1.476791268 podStartE2EDuration="2.552178159s" totalImagesPullingTime="1.075386891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:24:03.549510821 +0000 UTC m=+638.603932480" watchObservedRunningTime="2026-07-07 13:24:03.552178159 +0000 UTC m=+638.606599815" Jul 07 13:24:11.624355 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:11.624302 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:11.624798 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:11.624407 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:11.637088 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:11.637060 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:12.589293 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.589242 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pvdqb" Jul 07 13:24:12.718944 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.718905 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-stchm"] Jul 07 13:24:12.722452 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.722426 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.726547 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.726520 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 13:24:12.747872 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.747839 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-stchm"] Jul 07 13:24:12.874753 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.874658 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.874753 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.874701 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.874753 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.874729 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.874986 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.874767 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.874986 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.874812 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.874986 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.874831 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrrpm\" (UniqueName: \"kubernetes.io/projected/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-kube-api-access-mrrpm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.975562 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.975527 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.975562 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.975569 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.975786 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.975594 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.975786 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.975618 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.975786 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.975653 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.975786 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.975672 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrrpm\" (UniqueName: \"kubernetes.io/projected/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-kube-api-access-mrrpm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.975986 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.975961 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.976047 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.976023 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.976154 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.976136 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.978024 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.978002 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.978120 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.978105 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:12.987530 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:12.987496 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrrpm\" (UniqueName: \"kubernetes.io/projected/b42c1c4d-6519-48a0-9dc0-4feaeaa08a48-kube-api-access-mrrpm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-stchm\" (UID: \"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:13.033124 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:13.033091 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:13.205409 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:13.205376 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-stchm"] Jul 07 13:24:13.209168 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:24:13.209138 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb42c1c4d_6519_48a0_9dc0_4feaeaa08a48.slice/crio-1c882b2cc6167083a8157484ac90d8d64257b6bd80581bd04bb1f23b11c461e4 WatchSource:0}: Error finding container 1c882b2cc6167083a8157484ac90d8d64257b6bd80581bd04bb1f23b11c461e4: Status 404 returned error can't find the container with id 1c882b2cc6167083a8157484ac90d8d64257b6bd80581bd04bb1f23b11c461e4 Jul 07 13:24:13.584447 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:13.584411 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" event={"ID":"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48","Type":"ContainerStarted","Data":"edb51bf5a2b903b612f6e033a5f8c53fabf23e419ef0d0fe1b9519734b2a164f"} Jul 07 13:24:13.584681 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:13.584472 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" event={"ID":"b42c1c4d-6519-48a0-9dc0-4feaeaa08a48","Type":"ContainerStarted","Data":"1c882b2cc6167083a8157484ac90d8d64257b6bd80581bd04bb1f23b11c461e4"} Jul 07 13:24:13.608017 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:13.607963 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" podStartSLOduration=1.366413118 podStartE2EDuration="1.607944748s" totalImagesPullingTime="241.53163ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:24:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:24:13.606101499 +0000 UTC m=+648.660523178" watchObservedRunningTime="2026-07-07 13:24:13.607944748 +0000 UTC m=+648.662366404" Jul 07 13:24:23.033810 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:23.033767 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:23.034322 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:23.033829 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:23.046396 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:23.046365 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:24:23.646544 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:24:23.646509 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-stchm" Jul 07 13:30:00.657409 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:00.657323 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723850-7zgzt"] Jul 07 13:30:00.660605 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:00.660587 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" Jul 07 13:30:00.666926 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:00.666901 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:30:00.686499 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:00.686460 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723850-7zgzt"] Jul 07 13:30:00.784710 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:00.784663 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh6jt\" (UniqueName: \"kubernetes.io/projected/64dba76a-9892-4158-910a-3820c05f84e5-kube-api-access-rh6jt\") pod \"maas-api-key-cleanup-29723850-7zgzt\" (UID: \"64dba76a-9892-4158-910a-3820c05f84e5\") " pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" Jul 07 13:30:00.885967 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:00.885926 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh6jt\" (UniqueName: \"kubernetes.io/projected/64dba76a-9892-4158-910a-3820c05f84e5-kube-api-access-rh6jt\") pod \"maas-api-key-cleanup-29723850-7zgzt\" (UID: \"64dba76a-9892-4158-910a-3820c05f84e5\") " pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" Jul 07 13:30:00.896761 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:00.896725 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh6jt\" (UniqueName: \"kubernetes.io/projected/64dba76a-9892-4158-910a-3820c05f84e5-kube-api-access-rh6jt\") pod \"maas-api-key-cleanup-29723850-7zgzt\" (UID: \"64dba76a-9892-4158-910a-3820c05f84e5\") " pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" Jul 07 13:30:00.971619 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:00.971583 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" Jul 07 13:30:01.122392 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:01.122366 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723850-7zgzt"] Jul 07 13:30:01.124675 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:30:01.124639 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64dba76a_9892_4158_910a_3820c05f84e5.slice/crio-9b2512bc9b8aca052f144a4930bdc31647f169c9ebbf4bf4761f173ed87f8369 WatchSource:0}: Error finding container 9b2512bc9b8aca052f144a4930bdc31647f169c9ebbf4bf4761f173ed87f8369: Status 404 returned error can't find the container with id 9b2512bc9b8aca052f144a4930bdc31647f169c9ebbf4bf4761f173ed87f8369 Jul 07 13:30:01.126838 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:01.126815 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:30:01.387500 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:01.387413 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" event={"ID":"64dba76a-9892-4158-910a-3820c05f84e5","Type":"ContainerStarted","Data":"9b2512bc9b8aca052f144a4930bdc31647f169c9ebbf4bf4761f173ed87f8369"} Jul 07 13:30:05.411187 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:05.411143 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" event={"ID":"64dba76a-9892-4158-910a-3820c05f84e5","Type":"ContainerStarted","Data":"e9d12481a690f984c8d6ad85a3f8104389a8b32c12c8e5612d6e9203168306e3"} Jul 07 13:30:05.433859 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:05.433793 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" podStartSLOduration=2.071831162 podStartE2EDuration="5.433767771s" totalImagesPullingTime="3.361936609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:30:05.42940586 +0000 UTC m=+1000.483827540" watchObservedRunningTime="2026-07-07 13:30:05.433767771 +0000 UTC m=+1000.488189431" Jul 07 13:30:06.418012 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:06.417972 2574 generic.go:356] "Generic (PLEG): container finished" podID="64dba76a-9892-4158-910a-3820c05f84e5" containerID="e9d12481a690f984c8d6ad85a3f8104389a8b32c12c8e5612d6e9203168306e3" exitCode=0 Jul 07 13:30:06.418472 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:06.418061 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" event={"ID":"64dba76a-9892-4158-910a-3820c05f84e5","Type":"ContainerDied","Data":"e9d12481a690f984c8d6ad85a3f8104389a8b32c12c8e5612d6e9203168306e3"} Jul 07 13:30:07.566346 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:07.566318 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" Jul 07 13:30:07.649645 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:07.649607 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh6jt\" (UniqueName: \"kubernetes.io/projected/64dba76a-9892-4158-910a-3820c05f84e5-kube-api-access-rh6jt\") pod \"64dba76a-9892-4158-910a-3820c05f84e5\" (UID: \"64dba76a-9892-4158-910a-3820c05f84e5\") " Jul 07 13:30:07.651858 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:07.651827 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64dba76a-9892-4158-910a-3820c05f84e5-kube-api-access-rh6jt" (OuterVolumeSpecName: "kube-api-access-rh6jt") pod "64dba76a-9892-4158-910a-3820c05f84e5" (UID: "64dba76a-9892-4158-910a-3820c05f84e5"). InnerVolumeSpecName "kube-api-access-rh6jt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:30:07.750696 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:07.750648 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh6jt\" (UniqueName: \"kubernetes.io/projected/64dba76a-9892-4158-910a-3820c05f84e5-kube-api-access-rh6jt\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:30:08.432404 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:08.432377 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" Jul 07 13:30:08.432577 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:08.432372 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-7zgzt" event={"ID":"64dba76a-9892-4158-910a-3820c05f84e5","Type":"ContainerDied","Data":"9b2512bc9b8aca052f144a4930bdc31647f169c9ebbf4bf4761f173ed87f8369"} Jul 07 13:30:08.432577 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:30:08.432509 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b2512bc9b8aca052f144a4930bdc31647f169c9ebbf4bf4761f173ed87f8369" Jul 07 13:31:55.443145 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:55.443108 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4"] Jul 07 13:31:55.443740 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:55.443317 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" podUID="ca6747bf-09a6-4990-b1e8-ce15531497cd" containerName="manager" containerID="cri-o://819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4" gracePeriod=10 Jul 07 13:31:58.308989 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:58.308949 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:31:58.340465 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:58.340433 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca6747bf-09a6-4990-b1e8-ce15531497cd-extensions-socket-volume\") pod \"ca6747bf-09a6-4990-b1e8-ce15531497cd\" (UID: \"ca6747bf-09a6-4990-b1e8-ce15531497cd\") " Jul 07 13:31:58.340793 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:58.340767 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca6747bf-09a6-4990-b1e8-ce15531497cd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ca6747bf-09a6-4990-b1e8-ce15531497cd" (UID: "ca6747bf-09a6-4990-b1e8-ce15531497cd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:31:58.441429 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:58.441339 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfb6t\" (UniqueName: \"kubernetes.io/projected/ca6747bf-09a6-4990-b1e8-ce15531497cd-kube-api-access-pfb6t\") pod \"ca6747bf-09a6-4990-b1e8-ce15531497cd\" (UID: \"ca6747bf-09a6-4990-b1e8-ce15531497cd\") " Jul 07 13:31:58.441586 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:58.441507 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca6747bf-09a6-4990-b1e8-ce15531497cd-extensions-socket-volume\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:31:58.443452 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:58.443419 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca6747bf-09a6-4990-b1e8-ce15531497cd-kube-api-access-pfb6t" (OuterVolumeSpecName: "kube-api-access-pfb6t") pod "ca6747bf-09a6-4990-b1e8-ce15531497cd" (UID: "ca6747bf-09a6-4990-b1e8-ce15531497cd"). InnerVolumeSpecName "kube-api-access-pfb6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:31:58.542565 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:58.542521 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfb6t\" (UniqueName: \"kubernetes.io/projected/ca6747bf-09a6-4990-b1e8-ce15531497cd-kube-api-access-pfb6t\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:31:59.009565 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:59.009531 2574 generic.go:356] "Generic (PLEG): container finished" podID="ca6747bf-09a6-4990-b1e8-ce15531497cd" containerID="819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4" exitCode=0 Jul 07 13:31:59.009750 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:59.009573 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" event={"ID":"ca6747bf-09a6-4990-b1e8-ce15531497cd","Type":"ContainerDied","Data":"819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4"} Jul 07 13:31:59.009750 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:59.009578 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" Jul 07 13:31:59.009750 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:59.009617 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4" event={"ID":"ca6747bf-09a6-4990-b1e8-ce15531497cd","Type":"ContainerDied","Data":"54fe1c45139246126346b3e5d6a165765f83e8e0e82571cb827ebdbb89717413"} Jul 07 13:31:59.009750 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:59.009638 2574 scope.go:117] "RemoveContainer" containerID="819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4" Jul 07 13:31:59.020078 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:59.020056 2574 scope.go:117] "RemoveContainer" containerID="819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4" Jul 07 13:31:59.020378 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:31:59.020358 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4\": container with ID starting with 819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4 not found: ID does not exist" containerID="819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4" Jul 07 13:31:59.020439 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:59.020386 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4"} err="failed to get container status \"819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4\": rpc error: code = NotFound desc = could not find container \"819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4\": container with ID starting with 819ffe5222e9f0908d00a34409dec32e27ba7d43dd388b71f56fbc9351153ba4 not found: ID does not exist" Jul 07 13:31:59.034145 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:59.034101 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4"] Jul 07 13:31:59.038401 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:59.038360 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pk8s4"] Jul 07 13:31:59.433440 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:31:59.433349 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca6747bf-09a6-4990-b1e8-ce15531497cd" path="/var/lib/kubelet/pods/ca6747bf-09a6-4990-b1e8-ce15531497cd/volumes" Jul 07 13:32:56.711109 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.710885 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx"] Jul 07 13:32:56.712510 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.712477 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64dba76a-9892-4158-910a-3820c05f84e5" containerName="cleanup" Jul 07 13:32:56.712510 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.712510 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="64dba76a-9892-4158-910a-3820c05f84e5" containerName="cleanup" Jul 07 13:32:56.713070 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.712550 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca6747bf-09a6-4990-b1e8-ce15531497cd" containerName="manager" Jul 07 13:32:56.713070 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.712559 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca6747bf-09a6-4990-b1e8-ce15531497cd" containerName="manager" Jul 07 13:32:56.713070 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.712709 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca6747bf-09a6-4990-b1e8-ce15531497cd" containerName="manager" Jul 07 13:32:56.713070 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.712731 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="64dba76a-9892-4158-910a-3820c05f84e5" containerName="cleanup" Jul 07 13:32:56.716732 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.716697 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" Jul 07 13:32:56.719349 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.719322 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mszc8\"" Jul 07 13:32:56.725916 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.725889 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx"] Jul 07 13:32:56.825079 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.825037 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00c8bf29-1cd9-4de3-a19b-b55123a99c52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42vhx\" (UID: \"00c8bf29-1cd9-4de3-a19b-b55123a99c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" Jul 07 13:32:56.825306 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.825154 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpbl5\" (UniqueName: \"kubernetes.io/projected/00c8bf29-1cd9-4de3-a19b-b55123a99c52-kube-api-access-fpbl5\") pod \"kuadrant-operator-controller-manager-86878c769f-42vhx\" (UID: \"00c8bf29-1cd9-4de3-a19b-b55123a99c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" Jul 07 13:32:56.926592 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.926548 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpbl5\" (UniqueName: \"kubernetes.io/projected/00c8bf29-1cd9-4de3-a19b-b55123a99c52-kube-api-access-fpbl5\") pod \"kuadrant-operator-controller-manager-86878c769f-42vhx\" (UID: \"00c8bf29-1cd9-4de3-a19b-b55123a99c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" Jul 07 13:32:56.926782 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.926608 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00c8bf29-1cd9-4de3-a19b-b55123a99c52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42vhx\" (UID: \"00c8bf29-1cd9-4de3-a19b-b55123a99c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" Jul 07 13:32:56.926989 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.926971 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/00c8bf29-1cd9-4de3-a19b-b55123a99c52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42vhx\" (UID: \"00c8bf29-1cd9-4de3-a19b-b55123a99c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" Jul 07 13:32:56.940636 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:56.940604 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpbl5\" (UniqueName: \"kubernetes.io/projected/00c8bf29-1cd9-4de3-a19b-b55123a99c52-kube-api-access-fpbl5\") pod \"kuadrant-operator-controller-manager-86878c769f-42vhx\" (UID: \"00c8bf29-1cd9-4de3-a19b-b55123a99c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" Jul 07 13:32:57.028373 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:57.028280 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" Jul 07 13:32:57.201577 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:57.201552 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx"] Jul 07 13:32:57.204389 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:32:57.204356 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c8bf29_1cd9_4de3_a19b_b55123a99c52.slice/crio-ddffb31a62e664511d5f0e4c79d4aa15703a372751d2d9f6521554242cbfe469 WatchSource:0}: Error finding container ddffb31a62e664511d5f0e4c79d4aa15703a372751d2d9f6521554242cbfe469: Status 404 returned error can't find the container with id ddffb31a62e664511d5f0e4c79d4aa15703a372751d2d9f6521554242cbfe469 Jul 07 13:32:57.311940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:57.311826 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" event={"ID":"00c8bf29-1cd9-4de3-a19b-b55123a99c52","Type":"ContainerStarted","Data":"804c0668e845912ff4f880f6110c8df710b4839b25f2589812c991b7595335d5"} Jul 07 13:32:57.311940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:57.311886 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" event={"ID":"00c8bf29-1cd9-4de3-a19b-b55123a99c52","Type":"ContainerStarted","Data":"ddffb31a62e664511d5f0e4c79d4aa15703a372751d2d9f6521554242cbfe469"} Jul 07 13:32:57.311940 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:57.311906 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" Jul 07 13:32:57.341667 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:32:57.341606 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" podStartSLOduration=1.341587259 podStartE2EDuration="1.341587259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:32:57.340066023 +0000 UTC m=+1172.394487683" watchObservedRunningTime="2026-07-07 13:32:57.341587259 +0000 UTC m=+1172.396008916" Jul 07 13:33:08.319355 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:33:08.319323 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42vhx" Jul 07 13:39:19.644597 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.644558 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc"] Jul 07 13:39:19.648562 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.648535 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:19.651426 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.651383 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:39:19.651593 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.651518 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-31f94700-serving-cert\"" Jul 07 13:39:19.665104 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.665074 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc"] Jul 07 13:39:19.755008 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.754970 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-openshift-service-ca\") pod \"maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:19.755224 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.755032 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxwc8\" (UniqueName: \"kubernetes.io/projected/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-kube-api-access-qxwc8\") pod \"maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:19.755224 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.755063 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-maas-api-tls\") pod \"maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:19.856585 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.856531 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-openshift-service-ca\") pod \"maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:19.856585 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.856603 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxwc8\" (UniqueName: \"kubernetes.io/projected/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-kube-api-access-qxwc8\") pod \"maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:19.856889 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.856627 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-maas-api-tls\") pod \"maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:19.857227 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.857199 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-openshift-service-ca\") pod \"maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:19.859140 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.859117 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-maas-api-tls\") pod \"maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:19.870178 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.870140 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxwc8\" (UniqueName: \"kubernetes.io/projected/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-kube-api-access-qxwc8\") pod \"maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:19.961679 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:19.961630 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:20.113669 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:20.113643 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc"] Jul 07 13:39:20.115864 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:39:20.115835 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6bfa8b0_8447_402f_8bfe_4d1f63d2b6ae.slice/crio-776e835a81baefd85f75160ac9558b91694f3f185ab56bd0bb0670378cd9a7d4 WatchSource:0}: Error finding container 776e835a81baefd85f75160ac9558b91694f3f185ab56bd0bb0670378cd9a7d4: Status 404 returned error can't find the container with id 776e835a81baefd85f75160ac9558b91694f3f185ab56bd0bb0670378cd9a7d4 Jul 07 13:39:20.117348 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:20.117315 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:39:20.340918 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:20.340817 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" event={"ID":"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae","Type":"ContainerStarted","Data":"776e835a81baefd85f75160ac9558b91694f3f185ab56bd0bb0670378cd9a7d4"} Jul 07 13:39:22.357044 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:22.357004 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" event={"ID":"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae","Type":"ContainerStarted","Data":"8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003"} Jul 07 13:39:22.357515 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:22.357204 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:22.387961 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:22.387869 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" podStartSLOduration=1.736021816 podStartE2EDuration="3.387850455s" totalImagesPullingTime="1.651828639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:39:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:39:22.387545264 +0000 UTC m=+1557.441966941" watchObservedRunningTime="2026-07-07 13:39:22.387850455 +0000 UTC m=+1557.442272159" Jul 07 13:39:25.034776 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.034736 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc"] Jul 07 13:39:25.035429 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.035003 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" podUID="b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae" containerName="maas-api" containerID="cri-o://8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003" gracePeriod=30 Jul 07 13:39:25.288712 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.288638 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:25.375743 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.375690 2574 generic.go:356] "Generic (PLEG): container finished" podID="b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae" containerID="8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003" exitCode=0 Jul 07 13:39:25.375743 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.375733 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" Jul 07 13:39:25.376024 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.375775 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" event={"ID":"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae","Type":"ContainerDied","Data":"8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003"} Jul 07 13:39:25.376024 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.375834 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc" event={"ID":"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae","Type":"ContainerDied","Data":"776e835a81baefd85f75160ac9558b91694f3f185ab56bd0bb0670378cd9a7d4"} Jul 07 13:39:25.376024 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.375865 2574 scope.go:117] "RemoveContainer" containerID="8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003" Jul 07 13:39:25.385064 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.385046 2574 scope.go:117] "RemoveContainer" containerID="8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003" Jul 07 13:39:25.385402 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:39:25.385384 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003\": container with ID starting with 8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003 not found: ID does not exist" containerID="8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003" Jul 07 13:39:25.385459 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.385412 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003"} err="failed to get container status \"8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003\": rpc error: code = NotFound desc = could not find container \"8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003\": container with ID starting with 8697495ad5a997f329838d4c95674098faa2cdc298b4ab289abd8bf999acc003 not found: ID does not exist" Jul 07 13:39:25.406672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.406637 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxwc8\" (UniqueName: \"kubernetes.io/projected/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-kube-api-access-qxwc8\") pod \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " Jul 07 13:39:25.406869 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.406687 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-openshift-service-ca\") pod \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " Jul 07 13:39:25.406869 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.406745 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-maas-api-tls\") pod \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\" (UID: \"b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae\") " Jul 07 13:39:25.407021 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.406997 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae" (UID: "b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:39:25.408924 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.408899 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-kube-api-access-qxwc8" (OuterVolumeSpecName: "kube-api-access-qxwc8") pod "b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae" (UID: "b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae"). InnerVolumeSpecName "kube-api-access-qxwc8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:39:25.409013 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.408937 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae" (UID: "b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:39:25.507912 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.507864 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxwc8\" (UniqueName: \"kubernetes.io/projected/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-kube-api-access-qxwc8\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:39:25.507912 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.507905 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-openshift-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:39:25.507912 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.507916 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae-maas-api-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:39:25.697334 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.697275 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc"] Jul 07 13:39:25.699740 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:25.699711 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-31f94700-8648b6cc49-v9ncc"] Jul 07 13:39:27.432215 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:27.432175 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae" path="/var/lib/kubelet/pods/b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae/volumes" Jul 07 13:39:32.468868 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.468831 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4"] Jul 07 13:39:32.469387 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.469300 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae" containerName="maas-api" Jul 07 13:39:32.469387 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.469315 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae" containerName="maas-api" Jul 07 13:39:32.469387 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.469385 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6bfa8b0-8447-402f-8bfe-4d1f63d2b6ae" containerName="maas-api" Jul 07 13:39:32.472313 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.472292 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:32.474799 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.474773 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-de419fb7-serving-cert\"" Jul 07 13:39:32.474931 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.474824 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:39:32.484807 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.484776 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4"] Jul 07 13:39:32.573292 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.573232 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-openshift-service-ca\") pod \"maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:32.573487 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.573385 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-maas-api-tls\") pod \"maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:32.573487 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.573439 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nj5m\" (UniqueName: \"kubernetes.io/projected/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-kube-api-access-4nj5m\") pod \"maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:32.674023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.673989 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-maas-api-tls\") pod \"maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:32.674219 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.674032 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nj5m\" (UniqueName: \"kubernetes.io/projected/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-kube-api-access-4nj5m\") pod \"maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:32.674219 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.674069 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-openshift-service-ca\") pod \"maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:32.674739 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.674711 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-openshift-service-ca\") pod \"maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:32.676483 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.676459 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-maas-api-tls\") pod \"maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:32.683925 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.683899 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nj5m\" (UniqueName: \"kubernetes.io/projected/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-kube-api-access-4nj5m\") pod \"maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:32.785179 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.785059 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:32.932696 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:32.932618 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4"] Jul 07 13:39:32.934241 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:39:32.934208 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b57b77_1045_4726_9dd7_cf62eef0f9bd.slice/crio-dc2560dcad9dce28b5cfd42f7050f231cd33adb258fcb7ec0c15906fc6012633 WatchSource:0}: Error finding container dc2560dcad9dce28b5cfd42f7050f231cd33adb258fcb7ec0c15906fc6012633: Status 404 returned error can't find the container with id dc2560dcad9dce28b5cfd42f7050f231cd33adb258fcb7ec0c15906fc6012633 Jul 07 13:39:33.422296 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:33.422240 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" event={"ID":"f2b57b77-1045-4726-9dd7-cf62eef0f9bd","Type":"ContainerStarted","Data":"d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1"} Jul 07 13:39:33.422487 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:33.422316 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" event={"ID":"f2b57b77-1045-4726-9dd7-cf62eef0f9bd","Type":"ContainerStarted","Data":"dc2560dcad9dce28b5cfd42f7050f231cd33adb258fcb7ec0c15906fc6012633"} Jul 07 13:39:33.422487 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:33.422389 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:33.440070 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:33.440004 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" podStartSLOduration=1.061562714 podStartE2EDuration="1.439983035s" totalImagesPullingTime="378.420321ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:39:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:39:33.437920574 +0000 UTC m=+1568.492342232" watchObservedRunningTime="2026-07-07 13:39:33.439983035 +0000 UTC m=+1568.494404692" Jul 07 13:39:38.061440 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.061398 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4"] Jul 07 13:39:38.061903 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.061681 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" podUID="f2b57b77-1045-4726-9dd7-cf62eef0f9bd" containerName="maas-api" containerID="cri-o://d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1" gracePeriod=30 Jul 07 13:39:38.067690 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.067659 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:38.312516 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.312447 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:38.325672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.325640 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-maas-api-tls\") pod \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " Jul 07 13:39:38.325847 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.325689 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nj5m\" (UniqueName: \"kubernetes.io/projected/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-kube-api-access-4nj5m\") pod \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " Jul 07 13:39:38.325847 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.325718 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-openshift-service-ca\") pod \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\" (UID: \"f2b57b77-1045-4726-9dd7-cf62eef0f9bd\") " Jul 07 13:39:38.326261 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.326211 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f2b57b77-1045-4726-9dd7-cf62eef0f9bd" (UID: "f2b57b77-1045-4726-9dd7-cf62eef0f9bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:39:38.328041 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.328007 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-kube-api-access-4nj5m" (OuterVolumeSpecName: "kube-api-access-4nj5m") pod "f2b57b77-1045-4726-9dd7-cf62eef0f9bd" (UID: "f2b57b77-1045-4726-9dd7-cf62eef0f9bd"). InnerVolumeSpecName "kube-api-access-4nj5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:39:38.328041 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.328009 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f2b57b77-1045-4726-9dd7-cf62eef0f9bd" (UID: "f2b57b77-1045-4726-9dd7-cf62eef0f9bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:39:38.427181 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.427138 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-maas-api-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:39:38.427181 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.427176 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nj5m\" (UniqueName: \"kubernetes.io/projected/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-kube-api-access-4nj5m\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:39:38.427181 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.427187 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2b57b77-1045-4726-9dd7-cf62eef0f9bd-openshift-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:39:38.449687 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.449650 2574 generic.go:356] "Generic (PLEG): container finished" podID="f2b57b77-1045-4726-9dd7-cf62eef0f9bd" containerID="d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1" exitCode=0 Jul 07 13:39:38.449935 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.449687 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" event={"ID":"f2b57b77-1045-4726-9dd7-cf62eef0f9bd","Type":"ContainerDied","Data":"d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1"} Jul 07 13:39:38.449935 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.449716 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" Jul 07 13:39:38.449935 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.449746 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4" event={"ID":"f2b57b77-1045-4726-9dd7-cf62eef0f9bd","Type":"ContainerDied","Data":"dc2560dcad9dce28b5cfd42f7050f231cd33adb258fcb7ec0c15906fc6012633"} Jul 07 13:39:38.449935 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.449770 2574 scope.go:117] "RemoveContainer" containerID="d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1" Jul 07 13:39:38.460095 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.460074 2574 scope.go:117] "RemoveContainer" containerID="d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1" Jul 07 13:39:38.460447 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:39:38.460423 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1\": container with ID starting with d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1 not found: ID does not exist" containerID="d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1" Jul 07 13:39:38.460524 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.460455 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1"} err="failed to get container status \"d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1\": rpc error: code = NotFound desc = could not find container \"d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1\": container with ID starting with d8d1827be5ff2f357be704cec06db8e77cc8ab456db62639d17b49e251ed61d1 not found: ID does not exist" Jul 07 13:39:38.473910 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.473874 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4"] Jul 07 13:39:38.480681 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:38.480648 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-de419fb7-7568bdf9bf-5t7h4"] Jul 07 13:39:39.431740 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:39.431706 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2b57b77-1045-4726-9dd7-cf62eef0f9bd" path="/var/lib/kubelet/pods/f2b57b77-1045-4726-9dd7-cf62eef0f9bd/volumes" Jul 07 13:39:51.370044 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.370002 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz"] Jul 07 13:39:51.370490 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.370395 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b57b77-1045-4726-9dd7-cf62eef0f9bd" containerName="maas-api" Jul 07 13:39:51.370490 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.370408 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b57b77-1045-4726-9dd7-cf62eef0f9bd" containerName="maas-api" Jul 07 13:39:51.370490 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.370488 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2b57b77-1045-4726-9dd7-cf62eef0f9bd" containerName="maas-api" Jul 07 13:39:51.375017 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.374991 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:51.377507 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.377481 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-8c7f7fad-serving-cert\"" Jul 07 13:39:51.377663 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.377644 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:39:51.386861 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.386829 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz"] Jul 07 13:39:51.443826 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.443791 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77587d7c-f340-4adb-9b28-89f76fee0042-openshift-service-ca\") pod \"maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:51.444044 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.443849 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl29n\" (UniqueName: \"kubernetes.io/projected/77587d7c-f340-4adb-9b28-89f76fee0042-kube-api-access-xl29n\") pod \"maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:51.444044 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.443987 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77587d7c-f340-4adb-9b28-89f76fee0042-maas-api-tls\") pod \"maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:51.544724 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.544682 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77587d7c-f340-4adb-9b28-89f76fee0042-openshift-service-ca\") pod \"maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:51.544955 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.544737 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl29n\" (UniqueName: \"kubernetes.io/projected/77587d7c-f340-4adb-9b28-89f76fee0042-kube-api-access-xl29n\") pod \"maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:51.544955 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.544904 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77587d7c-f340-4adb-9b28-89f76fee0042-maas-api-tls\") pod \"maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:51.545506 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.545481 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77587d7c-f340-4adb-9b28-89f76fee0042-openshift-service-ca\") pod \"maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:51.547499 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.547476 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77587d7c-f340-4adb-9b28-89f76fee0042-maas-api-tls\") pod \"maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:51.565946 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.565906 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl29n\" (UniqueName: \"kubernetes.io/projected/77587d7c-f340-4adb-9b28-89f76fee0042-kube-api-access-xl29n\") pod \"maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:51.687803 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.687698 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:51.841846 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:51.841813 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz"] Jul 07 13:39:51.843772 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:39:51.843741 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77587d7c_f340_4adb_9b28_89f76fee0042.slice/crio-64d89f2b9c07cc18786d891c8d7ccea7f94229c5cfd345045827b39aa3dbe4e9 WatchSource:0}: Error finding container 64d89f2b9c07cc18786d891c8d7ccea7f94229c5cfd345045827b39aa3dbe4e9: Status 404 returned error can't find the container with id 64d89f2b9c07cc18786d891c8d7ccea7f94229c5cfd345045827b39aa3dbe4e9 Jul 07 13:39:52.527706 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:52.527664 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" event={"ID":"77587d7c-f340-4adb-9b28-89f76fee0042","Type":"ContainerStarted","Data":"6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9"} Jul 07 13:39:52.528103 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:52.527722 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" event={"ID":"77587d7c-f340-4adb-9b28-89f76fee0042","Type":"ContainerStarted","Data":"64d89f2b9c07cc18786d891c8d7ccea7f94229c5cfd345045827b39aa3dbe4e9"} Jul 07 13:39:52.528103 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:52.527786 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:52.551598 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:52.551519 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" podStartSLOduration=1.084294893 podStartE2EDuration="1.55149408s" totalImagesPullingTime="467.199187ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:39:52.549434366 +0000 UTC m=+1587.603856062" watchObservedRunningTime="2026-07-07 13:39:52.55149408 +0000 UTC m=+1587.605915739" Jul 07 13:39:56.100825 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.100783 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz"] Jul 07 13:39:56.101395 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.101054 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" podUID="77587d7c-f340-4adb-9b28-89f76fee0042" containerName="maas-api" containerID="cri-o://6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9" gracePeriod=30 Jul 07 13:39:56.364015 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.363940 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:56.488653 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.488614 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xl29n\" (UniqueName: \"kubernetes.io/projected/77587d7c-f340-4adb-9b28-89f76fee0042-kube-api-access-xl29n\") pod \"77587d7c-f340-4adb-9b28-89f76fee0042\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " Jul 07 13:39:56.488653 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.488666 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77587d7c-f340-4adb-9b28-89f76fee0042-openshift-service-ca\") pod \"77587d7c-f340-4adb-9b28-89f76fee0042\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " Jul 07 13:39:56.488946 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.488727 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77587d7c-f340-4adb-9b28-89f76fee0042-maas-api-tls\") pod \"77587d7c-f340-4adb-9b28-89f76fee0042\" (UID: \"77587d7c-f340-4adb-9b28-89f76fee0042\") " Jul 07 13:39:56.489182 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.489151 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77587d7c-f340-4adb-9b28-89f76fee0042-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "77587d7c-f340-4adb-9b28-89f76fee0042" (UID: "77587d7c-f340-4adb-9b28-89f76fee0042"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:39:56.490928 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.490895 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77587d7c-f340-4adb-9b28-89f76fee0042-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "77587d7c-f340-4adb-9b28-89f76fee0042" (UID: "77587d7c-f340-4adb-9b28-89f76fee0042"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:39:56.490928 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.490909 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77587d7c-f340-4adb-9b28-89f76fee0042-kube-api-access-xl29n" (OuterVolumeSpecName: "kube-api-access-xl29n") pod "77587d7c-f340-4adb-9b28-89f76fee0042" (UID: "77587d7c-f340-4adb-9b28-89f76fee0042"). InnerVolumeSpecName "kube-api-access-xl29n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:39:56.549731 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.549698 2574 generic.go:356] "Generic (PLEG): container finished" podID="77587d7c-f340-4adb-9b28-89f76fee0042" containerID="6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9" exitCode=0 Jul 07 13:39:56.549926 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.549737 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" Jul 07 13:39:56.549926 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.549776 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" event={"ID":"77587d7c-f340-4adb-9b28-89f76fee0042","Type":"ContainerDied","Data":"6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9"} Jul 07 13:39:56.549926 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.549826 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz" event={"ID":"77587d7c-f340-4adb-9b28-89f76fee0042","Type":"ContainerDied","Data":"64d89f2b9c07cc18786d891c8d7ccea7f94229c5cfd345045827b39aa3dbe4e9"} Jul 07 13:39:56.549926 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.549847 2574 scope.go:117] "RemoveContainer" containerID="6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9" Jul 07 13:39:56.559590 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.559563 2574 scope.go:117] "RemoveContainer" containerID="6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9" Jul 07 13:39:56.559887 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:39:56.559868 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9\": container with ID starting with 6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9 not found: ID does not exist" containerID="6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9" Jul 07 13:39:56.559937 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.559900 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9"} err="failed to get container status \"6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9\": rpc error: code = NotFound desc = could not find container \"6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9\": container with ID starting with 6bb994e4c16290263ce7caca23e9419bb3a570a4e3a8bc32ad1a340c229de5a9 not found: ID does not exist" Jul 07 13:39:56.570599 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.570570 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz"] Jul 07 13:39:56.574918 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.574895 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-8c7f7fad-7d8fbf6c76-j7qwz"] Jul 07 13:39:56.590058 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.590029 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77587d7c-f340-4adb-9b28-89f76fee0042-maas-api-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:39:56.590058 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.590058 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xl29n\" (UniqueName: \"kubernetes.io/projected/77587d7c-f340-4adb-9b28-89f76fee0042-kube-api-access-xl29n\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:39:56.590269 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:56.590072 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77587d7c-f340-4adb-9b28-89f76fee0042-openshift-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:39:57.432107 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:39:57.432072 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77587d7c-f340-4adb-9b28-89f76fee0042" path="/var/lib/kubelet/pods/77587d7c-f340-4adb-9b28-89f76fee0042/volumes" Jul 07 13:41:42.476954 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.476909 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb"] Jul 07 13:41:42.477474 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.477278 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77587d7c-f340-4adb-9b28-89f76fee0042" containerName="maas-api" Jul 07 13:41:42.477474 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.477291 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="77587d7c-f340-4adb-9b28-89f76fee0042" containerName="maas-api" Jul 07 13:41:42.477474 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.477353 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="77587d7c-f340-4adb-9b28-89f76fee0042" containerName="maas-api" Jul 07 13:41:42.480388 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.480369 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:42.485944 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.485914 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:41:42.486307 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.486288 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-bc796f0c-serving-cert\"" Jul 07 13:41:42.503919 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.503878 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb"] Jul 07 13:41:42.508904 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.508868 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs5bl\" (UniqueName: \"kubernetes.io/projected/4f9b648e-899d-4708-b805-afbdfbe19f19-kube-api-access-fs5bl\") pod \"maas-api-e2e-mt-bc796f0c-67756db968-whgjb\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:42.508904 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.508909 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f9b648e-899d-4708-b805-afbdfbe19f19-maas-api-tls\") pod \"maas-api-e2e-mt-bc796f0c-67756db968-whgjb\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:42.509198 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.509101 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9b648e-899d-4708-b805-afbdfbe19f19-openshift-service-ca\") pod \"maas-api-e2e-mt-bc796f0c-67756db968-whgjb\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:42.610565 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.610518 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9b648e-899d-4708-b805-afbdfbe19f19-openshift-service-ca\") pod \"maas-api-e2e-mt-bc796f0c-67756db968-whgjb\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:42.610565 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.610563 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs5bl\" (UniqueName: \"kubernetes.io/projected/4f9b648e-899d-4708-b805-afbdfbe19f19-kube-api-access-fs5bl\") pod \"maas-api-e2e-mt-bc796f0c-67756db968-whgjb\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:42.610804 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.610677 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f9b648e-899d-4708-b805-afbdfbe19f19-maas-api-tls\") pod \"maas-api-e2e-mt-bc796f0c-67756db968-whgjb\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:42.610804 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:41:42.610796 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-bc796f0c-serving-cert: secret "maas-api-e2e-mt-bc796f0c-serving-cert" not found Jul 07 13:41:42.610883 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:41:42.610862 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f9b648e-899d-4708-b805-afbdfbe19f19-maas-api-tls podName:4f9b648e-899d-4708-b805-afbdfbe19f19 nodeName:}" failed. No retries permitted until 2026-07-07 13:41:43.110844026 +0000 UTC m=+1698.165265675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4f9b648e-899d-4708-b805-afbdfbe19f19-maas-api-tls") pod "maas-api-e2e-mt-bc796f0c-67756db968-whgjb" (UID: "4f9b648e-899d-4708-b805-afbdfbe19f19") : secret "maas-api-e2e-mt-bc796f0c-serving-cert" not found Jul 07 13:41:42.611340 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.611320 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9b648e-899d-4708-b805-afbdfbe19f19-openshift-service-ca\") pod \"maas-api-e2e-mt-bc796f0c-67756db968-whgjb\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:42.624727 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:42.624690 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs5bl\" (UniqueName: \"kubernetes.io/projected/4f9b648e-899d-4708-b805-afbdfbe19f19-kube-api-access-fs5bl\") pod \"maas-api-e2e-mt-bc796f0c-67756db968-whgjb\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:43.114196 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:43.114157 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f9b648e-899d-4708-b805-afbdfbe19f19-maas-api-tls\") pod \"maas-api-e2e-mt-bc796f0c-67756db968-whgjb\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:43.116633 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:43.116604 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f9b648e-899d-4708-b805-afbdfbe19f19-maas-api-tls\") pod \"maas-api-e2e-mt-bc796f0c-67756db968-whgjb\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:43.391726 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:43.391619 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:43.557527 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:43.557501 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb"] Jul 07 13:41:43.559828 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:41:43.559800 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f9b648e_899d_4708_b805_afbdfbe19f19.slice/crio-7baf6d5a9feb412e998e702aea6c22956c28ad64dd259e8693e2f5a1919bd66f WatchSource:0}: Error finding container 7baf6d5a9feb412e998e702aea6c22956c28ad64dd259e8693e2f5a1919bd66f: Status 404 returned error can't find the container with id 7baf6d5a9feb412e998e702aea6c22956c28ad64dd259e8693e2f5a1919bd66f Jul 07 13:41:44.133933 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:44.133897 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" event={"ID":"4f9b648e-899d-4708-b805-afbdfbe19f19","Type":"ContainerStarted","Data":"19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200"} Jul 07 13:41:44.134120 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:44.133947 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" event={"ID":"4f9b648e-899d-4708-b805-afbdfbe19f19","Type":"ContainerStarted","Data":"7baf6d5a9feb412e998e702aea6c22956c28ad64dd259e8693e2f5a1919bd66f"} Jul 07 13:41:44.134120 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:44.133978 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:44.156229 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:44.156157 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" podStartSLOduration=1.703010471 podStartE2EDuration="2.156132816s" totalImagesPullingTime="453.122345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:41:44.155140535 +0000 UTC m=+1699.209562205" watchObservedRunningTime="2026-07-07 13:41:44.156132816 +0000 UTC m=+1699.210554475" Jul 07 13:41:50.145452 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:50.145419 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:54.474176 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.474135 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb"] Jul 07 13:41:54.474703 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.474407 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" podUID="4f9b648e-899d-4708-b805-afbdfbe19f19" containerName="maas-api" containerID="cri-o://19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200" gracePeriod=30 Jul 07 13:41:54.731941 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.731864 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:54.824049 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.824015 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f9b648e-899d-4708-b805-afbdfbe19f19-maas-api-tls\") pod \"4f9b648e-899d-4708-b805-afbdfbe19f19\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " Jul 07 13:41:54.824281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.824069 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs5bl\" (UniqueName: \"kubernetes.io/projected/4f9b648e-899d-4708-b805-afbdfbe19f19-kube-api-access-fs5bl\") pod \"4f9b648e-899d-4708-b805-afbdfbe19f19\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " Jul 07 13:41:54.824281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.824094 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9b648e-899d-4708-b805-afbdfbe19f19-openshift-service-ca\") pod \"4f9b648e-899d-4708-b805-afbdfbe19f19\" (UID: \"4f9b648e-899d-4708-b805-afbdfbe19f19\") " Jul 07 13:41:54.824590 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.824560 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f9b648e-899d-4708-b805-afbdfbe19f19-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f9b648e-899d-4708-b805-afbdfbe19f19" (UID: "4f9b648e-899d-4708-b805-afbdfbe19f19"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:41:54.826317 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.826265 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f9b648e-899d-4708-b805-afbdfbe19f19-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f9b648e-899d-4708-b805-afbdfbe19f19" (UID: "4f9b648e-899d-4708-b805-afbdfbe19f19"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:41:54.826409 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.826324 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f9b648e-899d-4708-b805-afbdfbe19f19-kube-api-access-fs5bl" (OuterVolumeSpecName: "kube-api-access-fs5bl") pod "4f9b648e-899d-4708-b805-afbdfbe19f19" (UID: "4f9b648e-899d-4708-b805-afbdfbe19f19"). InnerVolumeSpecName "kube-api-access-fs5bl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:41:54.925178 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.925130 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f9b648e-899d-4708-b805-afbdfbe19f19-maas-api-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:41:54.925178 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.925175 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs5bl\" (UniqueName: \"kubernetes.io/projected/4f9b648e-899d-4708-b805-afbdfbe19f19-kube-api-access-fs5bl\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:41:54.925430 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:54.925189 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9b648e-899d-4708-b805-afbdfbe19f19-openshift-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:41:55.194942 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:55.194850 2574 generic.go:356] "Generic (PLEG): container finished" podID="4f9b648e-899d-4708-b805-afbdfbe19f19" containerID="19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200" exitCode=0 Jul 07 13:41:55.194942 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:55.194896 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" Jul 07 13:41:55.195130 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:55.194938 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" event={"ID":"4f9b648e-899d-4708-b805-afbdfbe19f19","Type":"ContainerDied","Data":"19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200"} Jul 07 13:41:55.195130 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:55.194992 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb" event={"ID":"4f9b648e-899d-4708-b805-afbdfbe19f19","Type":"ContainerDied","Data":"7baf6d5a9feb412e998e702aea6c22956c28ad64dd259e8693e2f5a1919bd66f"} Jul 07 13:41:55.195130 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:55.195018 2574 scope.go:117] "RemoveContainer" containerID="19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200" Jul 07 13:41:55.204492 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:55.204471 2574 scope.go:117] "RemoveContainer" containerID="19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200" Jul 07 13:41:55.204833 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:41:55.204816 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200\": container with ID starting with 19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200 not found: ID does not exist" containerID="19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200" Jul 07 13:41:55.204871 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:55.204844 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200"} err="failed to get container status \"19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200\": rpc error: code = NotFound desc = could not find container \"19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200\": container with ID starting with 19365433cda2f8e3e5f8a8f1c1d18792fe734c716cb814d8f77cd81edb186200 not found: ID does not exist" Jul 07 13:41:55.224872 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:55.224828 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb"] Jul 07 13:41:55.226996 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:55.226970 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-bc796f0c-67756db968-whgjb"] Jul 07 13:41:55.432743 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:41:55.432709 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f9b648e-899d-4708-b805-afbdfbe19f19" path="/var/lib/kubelet/pods/4f9b648e-899d-4708-b805-afbdfbe19f19/volumes" Jul 07 13:42:24.595998 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.595959 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d"] Jul 07 13:42:24.596828 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.596488 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f9b648e-899d-4708-b805-afbdfbe19f19" containerName="maas-api" Jul 07 13:42:24.596828 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.596508 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f9b648e-899d-4708-b805-afbdfbe19f19" containerName="maas-api" Jul 07 13:42:24.596828 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.596607 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f9b648e-899d-4708-b805-afbdfbe19f19" containerName="maas-api" Jul 07 13:42:24.600014 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.599989 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:24.602371 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.602341 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:42:24.602520 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.602401 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-8880b7f7-serving-cert\"" Jul 07 13:42:24.608092 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.607464 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d"] Jul 07 13:42:24.688821 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.688778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-openshift-service-ca\") pod \"maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:24.688821 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.688831 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-maas-api-tls\") pod \"maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:24.689127 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.688897 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm69r\" (UniqueName: \"kubernetes.io/projected/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-kube-api-access-jm69r\") pod \"maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:24.789890 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.789838 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-openshift-service-ca\") pod \"maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:24.790081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.789913 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-maas-api-tls\") pod \"maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:24.790081 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.789942 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm69r\" (UniqueName: \"kubernetes.io/projected/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-kube-api-access-jm69r\") pod \"maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:24.790169 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:42:24.790071 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-8880b7f7-serving-cert: secret "maas-api-e2e-mt-8880b7f7-serving-cert" not found Jul 07 13:42:24.790169 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:42:24.790161 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-maas-api-tls podName:d2fbd7c9-f772-48ca-b189-ecdb165ebac3 nodeName:}" failed. No retries permitted until 2026-07-07 13:42:25.290138421 +0000 UTC m=+1740.344560061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-maas-api-tls") pod "maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" (UID: "d2fbd7c9-f772-48ca-b189-ecdb165ebac3") : secret "maas-api-e2e-mt-8880b7f7-serving-cert" not found Jul 07 13:42:24.790646 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.790627 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-openshift-service-ca\") pod \"maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:24.801053 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:24.801022 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm69r\" (UniqueName: \"kubernetes.io/projected/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-kube-api-access-jm69r\") pod \"maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:25.293903 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:25.293871 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-maas-api-tls\") pod \"maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:25.296395 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:25.296359 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-maas-api-tls\") pod \"maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:25.515240 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:25.515203 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:42:25.522651 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:25.522618 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:25.691502 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:25.691457 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d"] Jul 07 13:42:25.699700 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:42:25.699659 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2fbd7c9_f772_48ca_b189_ecdb165ebac3.slice/crio-5e2ee32f1ccb01f519cb6b5ff4d1c362a85512ab6e48871158f9cea591b12611 WatchSource:0}: Error finding container 5e2ee32f1ccb01f519cb6b5ff4d1c362a85512ab6e48871158f9cea591b12611: Status 404 returned error can't find the container with id 5e2ee32f1ccb01f519cb6b5ff4d1c362a85512ab6e48871158f9cea591b12611 Jul 07 13:42:26.360184 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:26.360143 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" event={"ID":"d2fbd7c9-f772-48ca-b189-ecdb165ebac3","Type":"ContainerStarted","Data":"23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391"} Jul 07 13:42:26.360396 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:26.360213 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" event={"ID":"d2fbd7c9-f772-48ca-b189-ecdb165ebac3","Type":"ContainerStarted","Data":"5e2ee32f1ccb01f519cb6b5ff4d1c362a85512ab6e48871158f9cea591b12611"} Jul 07 13:42:26.360396 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:26.360242 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:42:26.379512 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:26.379451 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" podStartSLOduration=1.860801188 podStartE2EDuration="2.379430839s" totalImagesPullingTime="518.629651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:42:26.3786481 +0000 UTC m=+1741.433069777" watchObservedRunningTime="2026-07-07 13:42:26.379430839 +0000 UTC m=+1741.433852524" Jul 07 13:42:32.371846 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:42:32.371810 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:43:07.091110 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.091064 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d"] Jul 07 13:43:07.091773 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.091335 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" podUID="d2fbd7c9-f772-48ca-b189-ecdb165ebac3" containerName="maas-api" containerID="cri-o://23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391" gracePeriod=30 Jul 07 13:43:07.341936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.341870 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:43:07.392028 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.391314 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-openshift-service-ca\") pod \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " Jul 07 13:43:07.393274 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.392399 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d2fbd7c9-f772-48ca-b189-ecdb165ebac3" (UID: "d2fbd7c9-f772-48ca-b189-ecdb165ebac3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:43:07.492893 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.492857 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-maas-api-tls\") pod \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " Jul 07 13:43:07.493071 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.492914 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm69r\" (UniqueName: \"kubernetes.io/projected/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-kube-api-access-jm69r\") pod \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\" (UID: \"d2fbd7c9-f772-48ca-b189-ecdb165ebac3\") " Jul 07 13:43:07.493115 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.493083 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-openshift-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:43:07.494908 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.494868 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d2fbd7c9-f772-48ca-b189-ecdb165ebac3" (UID: "d2fbd7c9-f772-48ca-b189-ecdb165ebac3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:43:07.495022 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.494975 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-kube-api-access-jm69r" (OuterVolumeSpecName: "kube-api-access-jm69r") pod "d2fbd7c9-f772-48ca-b189-ecdb165ebac3" (UID: "d2fbd7c9-f772-48ca-b189-ecdb165ebac3"). InnerVolumeSpecName "kube-api-access-jm69r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:43:07.591023 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.590978 2574 generic.go:356] "Generic (PLEG): container finished" podID="d2fbd7c9-f772-48ca-b189-ecdb165ebac3" containerID="23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391" exitCode=0 Jul 07 13:43:07.591223 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.591039 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" Jul 07 13:43:07.591223 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.591065 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" event={"ID":"d2fbd7c9-f772-48ca-b189-ecdb165ebac3","Type":"ContainerDied","Data":"23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391"} Jul 07 13:43:07.591223 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.591131 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d" event={"ID":"d2fbd7c9-f772-48ca-b189-ecdb165ebac3","Type":"ContainerDied","Data":"5e2ee32f1ccb01f519cb6b5ff4d1c362a85512ab6e48871158f9cea591b12611"} Jul 07 13:43:07.591223 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.591156 2574 scope.go:117] "RemoveContainer" containerID="23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391" Jul 07 13:43:07.594420 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.594396 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-maas-api-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:43:07.594513 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.594425 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm69r\" (UniqueName: \"kubernetes.io/projected/d2fbd7c9-f772-48ca-b189-ecdb165ebac3-kube-api-access-jm69r\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:43:07.601453 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.601433 2574 scope.go:117] "RemoveContainer" containerID="23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391" Jul 07 13:43:07.601787 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:43:07.601764 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391\": container with ID starting with 23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391 not found: ID does not exist" containerID="23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391" Jul 07 13:43:07.601841 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.601798 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391"} err="failed to get container status \"23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391\": rpc error: code = NotFound desc = could not find container \"23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391\": container with ID starting with 23e58e1321050e949b4c7640242543b9d00c4b241d040ab3e682c00f51288391 not found: ID does not exist" Jul 07 13:43:07.615198 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.615161 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d"] Jul 07 13:43:07.617336 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:07.617294 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-8880b7f7-754cf687f4-hk84d"] Jul 07 13:43:09.431815 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:43:09.431775 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2fbd7c9-f772-48ca-b189-ecdb165ebac3" path="/var/lib/kubelet/pods/d2fbd7c9-f772-48ca-b189-ecdb165ebac3/volumes" Jul 07 13:45:00.200708 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.200672 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723865-hsvwj"] Jul 07 13:45:00.201202 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.201003 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2fbd7c9-f772-48ca-b189-ecdb165ebac3" containerName="maas-api" Jul 07 13:45:00.201202 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.201015 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2fbd7c9-f772-48ca-b189-ecdb165ebac3" containerName="maas-api" Jul 07 13:45:00.201202 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.201084 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2fbd7c9-f772-48ca-b189-ecdb165ebac3" containerName="maas-api" Jul 07 13:45:00.203905 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.203886 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" Jul 07 13:45:00.217734 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.217708 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:45:00.229902 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.229867 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723865-hsvwj"] Jul 07 13:45:00.330607 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.330570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47v49\" (UniqueName: \"kubernetes.io/projected/beb2195c-d976-4a7d-bb5c-7524c64de3da-kube-api-access-47v49\") pod \"maas-api-key-cleanup-29723865-hsvwj\" (UID: \"beb2195c-d976-4a7d-bb5c-7524c64de3da\") " pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" Jul 07 13:45:00.431370 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.431329 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47v49\" (UniqueName: \"kubernetes.io/projected/beb2195c-d976-4a7d-bb5c-7524c64de3da-kube-api-access-47v49\") pod \"maas-api-key-cleanup-29723865-hsvwj\" (UID: \"beb2195c-d976-4a7d-bb5c-7524c64de3da\") " pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" Jul 07 13:45:00.443690 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.443655 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47v49\" (UniqueName: \"kubernetes.io/projected/beb2195c-d976-4a7d-bb5c-7524c64de3da-kube-api-access-47v49\") pod \"maas-api-key-cleanup-29723865-hsvwj\" (UID: \"beb2195c-d976-4a7d-bb5c-7524c64de3da\") " pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" Jul 07 13:45:00.513850 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.513804 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" Jul 07 13:45:00.679989 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:00.679964 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723865-hsvwj"] Jul 07 13:45:00.682418 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:45:00.682389 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeb2195c_d976_4a7d_bb5c_7524c64de3da.slice/crio-fa46c6186e5c475157922c4a0b105782a27fe91dfeb3af7d00687218679e0088 WatchSource:0}: Error finding container fa46c6186e5c475157922c4a0b105782a27fe91dfeb3af7d00687218679e0088: Status 404 returned error can't find the container with id fa46c6186e5c475157922c4a0b105782a27fe91dfeb3af7d00687218679e0088 Jul 07 13:45:01.204626 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:01.204521 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" event={"ID":"beb2195c-d976-4a7d-bb5c-7524c64de3da","Type":"ContainerStarted","Data":"d69b2ff5c6521bb7a1af3c9b85a3f076beb0ca3c42d41625e054ed28d820e646"} Jul 07 13:45:01.204626 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:01.204590 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" event={"ID":"beb2195c-d976-4a7d-bb5c-7524c64de3da","Type":"ContainerStarted","Data":"fa46c6186e5c475157922c4a0b105782a27fe91dfeb3af7d00687218679e0088"} Jul 07 13:45:01.259075 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:01.259012 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" podStartSLOduration=1.258993167 podStartE2EDuration="1.258993167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:45:01.254911372 +0000 UTC m=+1896.309333029" watchObservedRunningTime="2026-07-07 13:45:01.258993167 +0000 UTC m=+1896.313414823" Jul 07 13:45:02.211910 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:02.211877 2574 generic.go:356] "Generic (PLEG): container finished" podID="beb2195c-d976-4a7d-bb5c-7524c64de3da" containerID="d69b2ff5c6521bb7a1af3c9b85a3f076beb0ca3c42d41625e054ed28d820e646" exitCode=0 Jul 07 13:45:02.212344 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:02.211971 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" event={"ID":"beb2195c-d976-4a7d-bb5c-7524c64de3da","Type":"ContainerDied","Data":"d69b2ff5c6521bb7a1af3c9b85a3f076beb0ca3c42d41625e054ed28d820e646"} Jul 07 13:45:03.362594 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:03.362564 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" Jul 07 13:45:03.457494 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:03.457464 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47v49\" (UniqueName: \"kubernetes.io/projected/beb2195c-d976-4a7d-bb5c-7524c64de3da-kube-api-access-47v49\") pod \"beb2195c-d976-4a7d-bb5c-7524c64de3da\" (UID: \"beb2195c-d976-4a7d-bb5c-7524c64de3da\") " Jul 07 13:45:03.459534 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:03.459510 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beb2195c-d976-4a7d-bb5c-7524c64de3da-kube-api-access-47v49" (OuterVolumeSpecName: "kube-api-access-47v49") pod "beb2195c-d976-4a7d-bb5c-7524c64de3da" (UID: "beb2195c-d976-4a7d-bb5c-7524c64de3da"). InnerVolumeSpecName "kube-api-access-47v49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:45:03.558990 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:03.558887 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47v49\" (UniqueName: \"kubernetes.io/projected/beb2195c-d976-4a7d-bb5c-7524c64de3da-kube-api-access-47v49\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:45:04.224676 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:04.224638 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" Jul 07 13:45:04.224676 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:04.224667 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723865-hsvwj" event={"ID":"beb2195c-d976-4a7d-bb5c-7524c64de3da","Type":"ContainerDied","Data":"fa46c6186e5c475157922c4a0b105782a27fe91dfeb3af7d00687218679e0088"} Jul 07 13:45:04.224901 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:04.224718 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa46c6186e5c475157922c4a0b105782a27fe91dfeb3af7d00687218679e0088" Jul 07 13:45:16.646135 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.646092 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf"] Jul 07 13:45:16.646609 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.646452 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beb2195c-d976-4a7d-bb5c-7524c64de3da" containerName="cleanup" Jul 07 13:45:16.646609 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.646464 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="beb2195c-d976-4a7d-bb5c-7524c64de3da" containerName="cleanup" Jul 07 13:45:16.646609 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.646522 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="beb2195c-d976-4a7d-bb5c-7524c64de3da" containerName="cleanup" Jul 07 13:45:16.649609 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.649588 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:16.664272 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.664227 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:45:16.664457 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.664227 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-055c02-serving-cert\"" Jul 07 13:45:16.685946 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.685900 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf"] Jul 07 13:45:16.773178 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.773143 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-maas-api-tls\") pod \"maas-api-e2e-inf-a-055c02-744669c47c-2hfpf\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:16.773421 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.773208 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-openshift-service-ca\") pod \"maas-api-e2e-inf-a-055c02-744669c47c-2hfpf\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:16.773421 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.773240 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pl89\" (UniqueName: \"kubernetes.io/projected/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-kube-api-access-2pl89\") pod \"maas-api-e2e-inf-a-055c02-744669c47c-2hfpf\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:16.874076 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.874033 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-maas-api-tls\") pod \"maas-api-e2e-inf-a-055c02-744669c47c-2hfpf\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:16.874302 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.874133 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-openshift-service-ca\") pod \"maas-api-e2e-inf-a-055c02-744669c47c-2hfpf\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:16.874302 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.874175 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pl89\" (UniqueName: \"kubernetes.io/projected/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-kube-api-access-2pl89\") pod \"maas-api-e2e-inf-a-055c02-744669c47c-2hfpf\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:16.875008 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.874979 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-openshift-service-ca\") pod \"maas-api-e2e-inf-a-055c02-744669c47c-2hfpf\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:16.876808 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.876775 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-maas-api-tls\") pod \"maas-api-e2e-inf-a-055c02-744669c47c-2hfpf\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:16.907886 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.907801 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pl89\" (UniqueName: \"kubernetes.io/projected/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-kube-api-access-2pl89\") pod \"maas-api-e2e-inf-a-055c02-744669c47c-2hfpf\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:16.960042 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:16.959999 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:17.203377 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:17.203353 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf"] Jul 07 13:45:17.205183 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:45:17.205148 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee83c11_25e7_4fcd_8f3c_6f9ae3f13805.slice/crio-f52b47606d505ae86e8ea73cfe4c6a587da60eff44f994d9f3f6b9559e20532b WatchSource:0}: Error finding container f52b47606d505ae86e8ea73cfe4c6a587da60eff44f994d9f3f6b9559e20532b: Status 404 returned error can't find the container with id f52b47606d505ae86e8ea73cfe4c6a587da60eff44f994d9f3f6b9559e20532b Jul 07 13:45:17.206488 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:17.206461 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:45:17.294921 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:17.294883 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" event={"ID":"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805","Type":"ContainerStarted","Data":"f52b47606d505ae86e8ea73cfe4c6a587da60eff44f994d9f3f6b9559e20532b"} Jul 07 13:45:18.303047 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:18.303009 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" event={"ID":"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805","Type":"ContainerStarted","Data":"ca80a756253a1b4643d54e8ce1cd2bf9064cb0952e75829911beb3cbf9e2dd2e"} Jul 07 13:45:18.303525 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:18.303134 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:24.314718 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:24.314679 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:45:24.342757 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:24.342683 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" podStartSLOduration=7.799009949 podStartE2EDuration="8.342664078s" totalImagesPullingTime="543.654129ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:45:18.357806004 +0000 UTC m=+1913.412227682" watchObservedRunningTime="2026-07-07 13:45:24.342664078 +0000 UTC m=+1919.397085743" Jul 07 13:45:25.193889 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.193844 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd"] Jul 07 13:45:25.197525 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.197499 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:25.205618 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.205578 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-9285f1-serving-cert\"" Jul 07 13:45:25.235847 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.235802 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd"] Jul 07 13:45:25.351089 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.351054 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-maas-api-tls\") pod \"maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:25.351495 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.351107 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w57gv\" (UniqueName: \"kubernetes.io/projected/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-kube-api-access-w57gv\") pod \"maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:25.351495 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.351191 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:25.452590 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.452489 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-maas-api-tls\") pod \"maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:25.452590 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.452570 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w57gv\" (UniqueName: \"kubernetes.io/projected/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-kube-api-access-w57gv\") pod \"maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:25.452835 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.452613 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:25.453354 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.453313 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:25.457512 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.457485 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-9285f1-serving-cert\"" Jul 07 13:45:25.466000 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.465970 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-maas-api-tls\") pod \"maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:25.468887 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.468860 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w57gv\" (UniqueName: \"kubernetes.io/projected/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-kube-api-access-w57gv\") pod \"maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:25.511027 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.510977 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:25.700826 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:25.700798 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd"] Jul 07 13:45:25.703184 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:45:25.703111 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4ce562_fd2f_4d4d_81a7_2d57530728c7.slice/crio-3f6bf56c99dfa0342accb163a333a9ed5a7c8ba83192c2d6098a473c2baf9769 WatchSource:0}: Error finding container 3f6bf56c99dfa0342accb163a333a9ed5a7c8ba83192c2d6098a473c2baf9769: Status 404 returned error can't find the container with id 3f6bf56c99dfa0342accb163a333a9ed5a7c8ba83192c2d6098a473c2baf9769 Jul 07 13:45:26.346881 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:26.346844 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" event={"ID":"fd4ce562-fd2f-4d4d-81a7-2d57530728c7","Type":"ContainerStarted","Data":"c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82"} Jul 07 13:45:26.347065 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:26.346905 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" event={"ID":"fd4ce562-fd2f-4d4d-81a7-2d57530728c7","Type":"ContainerStarted","Data":"3f6bf56c99dfa0342accb163a333a9ed5a7c8ba83192c2d6098a473c2baf9769"} Jul 07 13:45:26.347065 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:26.346933 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:45:26.390360 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:26.390236 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" podStartSLOduration=0.969046801 podStartE2EDuration="1.390218289s" totalImagesPullingTime="421.171488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:45:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:45:26.389018123 +0000 UTC m=+1921.443439781" watchObservedRunningTime="2026-07-07 13:45:26.390218289 +0000 UTC m=+1921.444639946" Jul 07 13:45:32.357397 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:45:32.357366 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:46:45.600442 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:45.600358 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf"] Jul 07 13:46:45.601042 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:45.600571 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" podUID="bee83c11-25e7-4fcd-8f3c-6f9ae3f13805" containerName="maas-api" containerID="cri-o://ca80a756253a1b4643d54e8ce1cd2bf9064cb0952e75829911beb3cbf9e2dd2e" gracePeriod=30 Jul 07 13:46:45.764516 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:45.764480 2574 generic.go:356] "Generic (PLEG): container finished" podID="bee83c11-25e7-4fcd-8f3c-6f9ae3f13805" containerID="ca80a756253a1b4643d54e8ce1cd2bf9064cb0952e75829911beb3cbf9e2dd2e" exitCode=0 Jul 07 13:46:45.764516 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:45.764509 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" event={"ID":"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805","Type":"ContainerDied","Data":"ca80a756253a1b4643d54e8ce1cd2bf9064cb0952e75829911beb3cbf9e2dd2e"} Jul 07 13:46:45.857660 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:45.857581 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:46:45.977152 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:45.977107 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-maas-api-tls\") pod \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " Jul 07 13:46:45.977383 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:45.977193 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pl89\" (UniqueName: \"kubernetes.io/projected/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-kube-api-access-2pl89\") pod \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " Jul 07 13:46:45.977383 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:45.977231 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-openshift-service-ca\") pod \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\" (UID: \"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805\") " Jul 07 13:46:45.977669 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:45.977643 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bee83c11-25e7-4fcd-8f3c-6f9ae3f13805" (UID: "bee83c11-25e7-4fcd-8f3c-6f9ae3f13805"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:46:45.979475 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:45.979436 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bee83c11-25e7-4fcd-8f3c-6f9ae3f13805" (UID: "bee83c11-25e7-4fcd-8f3c-6f9ae3f13805"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:46:45.979582 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:45.979496 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-kube-api-access-2pl89" (OuterVolumeSpecName: "kube-api-access-2pl89") pod "bee83c11-25e7-4fcd-8f3c-6f9ae3f13805" (UID: "bee83c11-25e7-4fcd-8f3c-6f9ae3f13805"). InnerVolumeSpecName "kube-api-access-2pl89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:46:46.078675 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:46.078625 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-maas-api-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:46:46.078675 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:46.078665 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pl89\" (UniqueName: \"kubernetes.io/projected/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-kube-api-access-2pl89\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:46:46.078675 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:46.078675 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805-openshift-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:46:46.771054 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:46.771015 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" event={"ID":"bee83c11-25e7-4fcd-8f3c-6f9ae3f13805","Type":"ContainerDied","Data":"f52b47606d505ae86e8ea73cfe4c6a587da60eff44f994d9f3f6b9559e20532b"} Jul 07 13:46:46.771610 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:46.771076 2574 scope.go:117] "RemoveContainer" containerID="ca80a756253a1b4643d54e8ce1cd2bf9064cb0952e75829911beb3cbf9e2dd2e" Jul 07 13:46:46.771610 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:46.771026 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf" Jul 07 13:46:46.797077 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:46.797029 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf"] Jul 07 13:46:46.804320 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:46.804287 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-055c02-744669c47c-2hfpf"] Jul 07 13:46:47.432033 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:46:47.431997 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bee83c11-25e7-4fcd-8f3c-6f9ae3f13805" path="/var/lib/kubelet/pods/bee83c11-25e7-4fcd-8f3c-6f9ae3f13805/volumes" Jul 07 13:47:29.580822 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.580781 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd"] Jul 07 13:47:29.581348 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.581041 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" podUID="fd4ce562-fd2f-4d4d-81a7-2d57530728c7" containerName="maas-api" containerID="cri-o://c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82" gracePeriod=30 Jul 07 13:47:29.832099 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.832022 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:47:29.943060 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.943022 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w57gv\" (UniqueName: \"kubernetes.io/projected/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-kube-api-access-w57gv\") pod \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " Jul 07 13:47:29.943278 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.943085 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-maas-api-tls\") pod \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " Jul 07 13:47:29.943278 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.943108 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-openshift-service-ca\") pod \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\" (UID: \"fd4ce562-fd2f-4d4d-81a7-2d57530728c7\") " Jul 07 13:47:29.943562 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.943528 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd4ce562-fd2f-4d4d-81a7-2d57530728c7" (UID: "fd4ce562-fd2f-4d4d-81a7-2d57530728c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:47:29.945396 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.945366 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-kube-api-access-w57gv" (OuterVolumeSpecName: "kube-api-access-w57gv") pod "fd4ce562-fd2f-4d4d-81a7-2d57530728c7" (UID: "fd4ce562-fd2f-4d4d-81a7-2d57530728c7"). InnerVolumeSpecName "kube-api-access-w57gv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:47:29.945509 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.945407 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd4ce562-fd2f-4d4d-81a7-2d57530728c7" (UID: "fd4ce562-fd2f-4d4d-81a7-2d57530728c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:47:29.994002 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.993960 2574 generic.go:356] "Generic (PLEG): container finished" podID="fd4ce562-fd2f-4d4d-81a7-2d57530728c7" containerID="c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82" exitCode=0 Jul 07 13:47:29.994002 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.994006 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" Jul 07 13:47:29.994235 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.994049 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" event={"ID":"fd4ce562-fd2f-4d4d-81a7-2d57530728c7","Type":"ContainerDied","Data":"c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82"} Jul 07 13:47:29.994235 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.994104 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd" event={"ID":"fd4ce562-fd2f-4d4d-81a7-2d57530728c7","Type":"ContainerDied","Data":"3f6bf56c99dfa0342accb163a333a9ed5a7c8ba83192c2d6098a473c2baf9769"} Jul 07 13:47:29.994235 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:29.994127 2574 scope.go:117] "RemoveContainer" containerID="c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82" Jul 07 13:47:30.003998 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:30.003977 2574 scope.go:117] "RemoveContainer" containerID="c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82" Jul 07 13:47:30.004313 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:47:30.004291 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82\": container with ID starting with c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82 not found: ID does not exist" containerID="c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82" Jul 07 13:47:30.004372 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:30.004326 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82"} err="failed to get container status \"c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82\": rpc error: code = NotFound desc = could not find container \"c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82\": container with ID starting with c4854ae34c82e889fe30369e8894a60e57d0e0aa476259f56577e1837ca46d82 not found: ID does not exist" Jul 07 13:47:30.024654 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:30.024613 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd"] Jul 07 13:47:30.032528 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:30.032493 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9285f1-9d9865ffb-62xjd"] Jul 07 13:47:30.044523 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:30.044490 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w57gv\" (UniqueName: \"kubernetes.io/projected/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-kube-api-access-w57gv\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:47:30.044523 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:30.044518 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-maas-api-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:47:30.044523 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:30.044531 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ce562-fd2f-4d4d-81a7-2d57530728c7-openshift-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:47:31.431209 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:31.431167 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd4ce562-fd2f-4d4d-81a7-2d57530728c7" path="/var/lib/kubelet/pods/fd4ce562-fd2f-4d4d-81a7-2d57530728c7/volumes" Jul 07 13:47:40.639163 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.639117 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs"] Jul 07 13:47:40.639656 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.639521 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd4ce562-fd2f-4d4d-81a7-2d57530728c7" containerName="maas-api" Jul 07 13:47:40.639656 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.639535 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd4ce562-fd2f-4d4d-81a7-2d57530728c7" containerName="maas-api" Jul 07 13:47:40.639656 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.639549 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bee83c11-25e7-4fcd-8f3c-6f9ae3f13805" containerName="maas-api" Jul 07 13:47:40.639656 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.639554 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bee83c11-25e7-4fcd-8f3c-6f9ae3f13805" containerName="maas-api" Jul 07 13:47:40.639656 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.639613 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bee83c11-25e7-4fcd-8f3c-6f9ae3f13805" containerName="maas-api" Jul 07 13:47:40.639656 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.639628 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd4ce562-fd2f-4d4d-81a7-2d57530728c7" containerName="maas-api" Jul 07 13:47:40.644042 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.644019 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:40.648318 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.648291 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:47:40.652881 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.652862 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-843413-serving-cert\"" Jul 07 13:47:40.671146 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.671110 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs"] Jul 07 13:47:40.739686 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.739640 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/889e1d75-9a8a-4355-80f5-9f61cbc70738-openshift-service-ca\") pod \"maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:40.739686 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.739684 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/889e1d75-9a8a-4355-80f5-9f61cbc70738-maas-api-tls\") pod \"maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:40.739969 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.739775 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m288\" (UniqueName: \"kubernetes.io/projected/889e1d75-9a8a-4355-80f5-9f61cbc70738-kube-api-access-6m288\") pod \"maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:40.840790 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.840752 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/889e1d75-9a8a-4355-80f5-9f61cbc70738-maas-api-tls\") pod \"maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:40.841030 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.840834 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m288\" (UniqueName: \"kubernetes.io/projected/889e1d75-9a8a-4355-80f5-9f61cbc70738-kube-api-access-6m288\") pod \"maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:40.841030 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.840917 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/889e1d75-9a8a-4355-80f5-9f61cbc70738-openshift-service-ca\") pod \"maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:40.841661 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.841632 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/889e1d75-9a8a-4355-80f5-9f61cbc70738-openshift-service-ca\") pod \"maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:40.843559 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.843531 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/889e1d75-9a8a-4355-80f5-9f61cbc70738-maas-api-tls\") pod \"maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:40.870857 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.870828 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m288\" (UniqueName: \"kubernetes.io/projected/889e1d75-9a8a-4355-80f5-9f61cbc70738-kube-api-access-6m288\") pod \"maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:40.954907 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:40.954867 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:41.110892 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:41.110860 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs"] Jul 07 13:47:41.111806 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:47:41.111773 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889e1d75_9a8a_4355_80f5_9f61cbc70738.slice/crio-c624ad791249c961168e2f77b5fde74e6c0e06178fe56e6076579c454ee17a7b WatchSource:0}: Error finding container c624ad791249c961168e2f77b5fde74e6c0e06178fe56e6076579c454ee17a7b: Status 404 returned error can't find the container with id c624ad791249c961168e2f77b5fde74e6c0e06178fe56e6076579c454ee17a7b Jul 07 13:47:42.063303 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:42.063232 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" event={"ID":"889e1d75-9a8a-4355-80f5-9f61cbc70738","Type":"ContainerStarted","Data":"26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101"} Jul 07 13:47:42.063830 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:42.063426 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:42.063830 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:42.063452 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" event={"ID":"889e1d75-9a8a-4355-80f5-9f61cbc70738","Type":"ContainerStarted","Data":"c624ad791249c961168e2f77b5fde74e6c0e06178fe56e6076579c454ee17a7b"} Jul 07 13:47:42.084454 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:42.084395 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" podStartSLOduration=1.694121657 podStartE2EDuration="2.084375677s" totalImagesPullingTime="390.25402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:47:42.082158422 +0000 UTC m=+2057.136580079" watchObservedRunningTime="2026-07-07 13:47:42.084375677 +0000 UTC m=+2057.138797332" Jul 07 13:47:48.074795 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.074712 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:47:48.485213 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.485174 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx"] Jul 07 13:47:48.489768 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.489746 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:48.515590 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:47:48.515543 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-api-b-fe52df-688c457b8-4dvvx\" is forbidden: User \"system:node:ip-10-0-143-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-95.ec2.internal' and this object" podUID="01c4c94c-4c5a-4996-9302-90088304e4c4" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:48.515590 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:47:48.515564 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-api-b-fe52df-serving-cert\" is forbidden: User \"system:node:ip-10-0-143-95.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-95.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-fe52df-serving-cert\"" type="*v1.Secret" Jul 07 13:47:48.611210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.611163 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sxmt\" (UniqueName: \"kubernetes.io/projected/01c4c94c-4c5a-4996-9302-90088304e4c4-kube-api-access-4sxmt\") pod \"maas-api-e2e-api-b-fe52df-688c457b8-4dvvx\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:48.611210 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.611209 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c4c94c-4c5a-4996-9302-90088304e4c4-maas-api-tls\") pod \"maas-api-e2e-api-b-fe52df-688c457b8-4dvvx\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:48.611456 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.611274 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c4c94c-4c5a-4996-9302-90088304e4c4-openshift-service-ca\") pod \"maas-api-e2e-api-b-fe52df-688c457b8-4dvvx\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:48.712035 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.711983 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sxmt\" (UniqueName: \"kubernetes.io/projected/01c4c94c-4c5a-4996-9302-90088304e4c4-kube-api-access-4sxmt\") pod \"maas-api-e2e-api-b-fe52df-688c457b8-4dvvx\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:48.712236 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.712049 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c4c94c-4c5a-4996-9302-90088304e4c4-maas-api-tls\") pod \"maas-api-e2e-api-b-fe52df-688c457b8-4dvvx\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:48.712236 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.712104 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c4c94c-4c5a-4996-9302-90088304e4c4-openshift-service-ca\") pod \"maas-api-e2e-api-b-fe52df-688c457b8-4dvvx\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:48.713032 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.713003 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c4c94c-4c5a-4996-9302-90088304e4c4-openshift-service-ca\") pod \"maas-api-e2e-api-b-fe52df-688c457b8-4dvvx\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:48.713725 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.713704 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx"] Jul 07 13:47:48.739867 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:48.739783 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sxmt\" (UniqueName: \"kubernetes.io/projected/01c4c94c-4c5a-4996-9302-90088304e4c4-kube-api-access-4sxmt\") pod \"maas-api-e2e-api-b-fe52df-688c457b8-4dvvx\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:49.713237 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:47:49.713198 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-fe52df-serving-cert: failed to sync secret cache: timed out waiting for the condition Jul 07 13:47:49.713664 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:47:49.713298 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01c4c94c-4c5a-4996-9302-90088304e4c4-maas-api-tls podName:01c4c94c-4c5a-4996-9302-90088304e4c4 nodeName:}" failed. No retries permitted until 2026-07-07 13:47:50.213278354 +0000 UTC m=+2065.267699994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/01c4c94c-4c5a-4996-9302-90088304e4c4-maas-api-tls") pod "maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" (UID: "01c4c94c-4c5a-4996-9302-90088304e4c4") : failed to sync secret cache: timed out waiting for the condition Jul 07 13:47:50.059007 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:50.058908 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-fe52df-serving-cert\"" Jul 07 13:47:50.223324 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:50.223275 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c4c94c-4c5a-4996-9302-90088304e4c4-maas-api-tls\") pod \"maas-api-e2e-api-b-fe52df-688c457b8-4dvvx\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:50.225773 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:50.225741 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c4c94c-4c5a-4996-9302-90088304e4c4-maas-api-tls\") pod \"maas-api-e2e-api-b-fe52df-688c457b8-4dvvx\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:50.300093 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:50.300040 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:50.457131 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:50.457103 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx"] Jul 07 13:47:50.458923 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:47:50.458895 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c4c94c_4c5a_4996_9302_90088304e4c4.slice/crio-21e9f6c5d8d4533664ff0aaae7135fb94efe51ac753d854ca7fdaa98afc4ed7a WatchSource:0}: Error finding container 21e9f6c5d8d4533664ff0aaae7135fb94efe51ac753d854ca7fdaa98afc4ed7a: Status 404 returned error can't find the container with id 21e9f6c5d8d4533664ff0aaae7135fb94efe51ac753d854ca7fdaa98afc4ed7a Jul 07 13:47:51.119115 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:51.119000 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" event={"ID":"01c4c94c-4c5a-4996-9302-90088304e4c4","Type":"ContainerStarted","Data":"8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b"} Jul 07 13:47:51.119115 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:51.119065 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" event={"ID":"01c4c94c-4c5a-4996-9302-90088304e4c4","Type":"ContainerStarted","Data":"21e9f6c5d8d4533664ff0aaae7135fb94efe51ac753d854ca7fdaa98afc4ed7a"} Jul 07 13:47:51.119612 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:51.119162 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:47:51.139096 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:51.139031 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" podStartSLOduration=2.78177404 podStartE2EDuration="3.139011561s" totalImagesPullingTime="357.237521ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:47:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:47:51.137319804 +0000 UTC m=+2066.191741477" watchObservedRunningTime="2026-07-07 13:47:51.139011561 +0000 UTC m=+2066.193433219" Jul 07 13:47:57.130011 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:47:57.129977 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:48:02.376007 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.375962 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs"] Jul 07 13:48:02.376601 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.376206 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" podUID="889e1d75-9a8a-4355-80f5-9f61cbc70738" containerName="maas-api" containerID="cri-o://26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101" gracePeriod=30 Jul 07 13:48:02.625605 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.625578 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:48:02.743712 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.743678 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m288\" (UniqueName: \"kubernetes.io/projected/889e1d75-9a8a-4355-80f5-9f61cbc70738-kube-api-access-6m288\") pod \"889e1d75-9a8a-4355-80f5-9f61cbc70738\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " Jul 07 13:48:02.743904 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.743784 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/889e1d75-9a8a-4355-80f5-9f61cbc70738-maas-api-tls\") pod \"889e1d75-9a8a-4355-80f5-9f61cbc70738\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " Jul 07 13:48:02.743904 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.743811 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/889e1d75-9a8a-4355-80f5-9f61cbc70738-openshift-service-ca\") pod \"889e1d75-9a8a-4355-80f5-9f61cbc70738\" (UID: \"889e1d75-9a8a-4355-80f5-9f61cbc70738\") " Jul 07 13:48:02.744236 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.744210 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/889e1d75-9a8a-4355-80f5-9f61cbc70738-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "889e1d75-9a8a-4355-80f5-9f61cbc70738" (UID: "889e1d75-9a8a-4355-80f5-9f61cbc70738"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:48:02.746046 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.746017 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/889e1d75-9a8a-4355-80f5-9f61cbc70738-kube-api-access-6m288" (OuterVolumeSpecName: "kube-api-access-6m288") pod "889e1d75-9a8a-4355-80f5-9f61cbc70738" (UID: "889e1d75-9a8a-4355-80f5-9f61cbc70738"). InnerVolumeSpecName "kube-api-access-6m288". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:48:02.746151 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.746020 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/889e1d75-9a8a-4355-80f5-9f61cbc70738-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "889e1d75-9a8a-4355-80f5-9f61cbc70738" (UID: "889e1d75-9a8a-4355-80f5-9f61cbc70738"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:48:02.844838 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.844798 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/889e1d75-9a8a-4355-80f5-9f61cbc70738-maas-api-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:48:02.844838 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.844831 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/889e1d75-9a8a-4355-80f5-9f61cbc70738-openshift-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:48:02.844838 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:02.844841 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m288\" (UniqueName: \"kubernetes.io/projected/889e1d75-9a8a-4355-80f5-9f61cbc70738-kube-api-access-6m288\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:48:03.187739 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:03.187633 2574 generic.go:356] "Generic (PLEG): container finished" podID="889e1d75-9a8a-4355-80f5-9f61cbc70738" containerID="26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101" exitCode=0 Jul 07 13:48:03.187739 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:03.187682 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" Jul 07 13:48:03.187739 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:03.187703 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" event={"ID":"889e1d75-9a8a-4355-80f5-9f61cbc70738","Type":"ContainerDied","Data":"26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101"} Jul 07 13:48:03.188041 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:03.187757 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs" event={"ID":"889e1d75-9a8a-4355-80f5-9f61cbc70738","Type":"ContainerDied","Data":"c624ad791249c961168e2f77b5fde74e6c0e06178fe56e6076579c454ee17a7b"} Jul 07 13:48:03.188041 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:03.187786 2574 scope.go:117] "RemoveContainer" containerID="26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101" Jul 07 13:48:03.197679 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:03.197658 2574 scope.go:117] "RemoveContainer" containerID="26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101" Jul 07 13:48:03.197965 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:48:03.197936 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101\": container with ID starting with 26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101 not found: ID does not exist" containerID="26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101" Jul 07 13:48:03.198019 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:03.197975 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101"} err="failed to get container status \"26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101\": rpc error: code = NotFound desc = could not find container \"26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101\": container with ID starting with 26d64c2e2283af1b791e09de648c71a9cc189b25cbf8cf9b873ffa27ef7f2101 not found: ID does not exist" Jul 07 13:48:03.219133 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:03.219092 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs"] Jul 07 13:48:03.225679 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:03.225649 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-843413-7d56fc7ffb-k7lqs"] Jul 07 13:48:03.431739 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:03.431701 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="889e1d75-9a8a-4355-80f5-9f61cbc70738" path="/var/lib/kubelet/pods/889e1d75-9a8a-4355-80f5-9f61cbc70738/volumes" Jul 07 13:48:09.934461 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:09.934428 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx"] Jul 07 13:48:09.934959 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:09.934679 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" podUID="01c4c94c-4c5a-4996-9302-90088304e4c4" containerName="maas-api" containerID="cri-o://8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b" gracePeriod=30 Jul 07 13:48:10.185693 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.185622 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:48:10.206190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.206149 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c4c94c-4c5a-4996-9302-90088304e4c4-openshift-service-ca\") pod \"01c4c94c-4c5a-4996-9302-90088304e4c4\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " Jul 07 13:48:10.206190 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.206196 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c4c94c-4c5a-4996-9302-90088304e4c4-maas-api-tls\") pod \"01c4c94c-4c5a-4996-9302-90088304e4c4\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " Jul 07 13:48:10.206500 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.206284 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sxmt\" (UniqueName: \"kubernetes.io/projected/01c4c94c-4c5a-4996-9302-90088304e4c4-kube-api-access-4sxmt\") pod \"01c4c94c-4c5a-4996-9302-90088304e4c4\" (UID: \"01c4c94c-4c5a-4996-9302-90088304e4c4\") " Jul 07 13:48:10.206688 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.206647 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01c4c94c-4c5a-4996-9302-90088304e4c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "01c4c94c-4c5a-4996-9302-90088304e4c4" (UID: "01c4c94c-4c5a-4996-9302-90088304e4c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:48:10.208499 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.208471 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01c4c94c-4c5a-4996-9302-90088304e4c4-kube-api-access-4sxmt" (OuterVolumeSpecName: "kube-api-access-4sxmt") pod "01c4c94c-4c5a-4996-9302-90088304e4c4" (UID: "01c4c94c-4c5a-4996-9302-90088304e4c4"). InnerVolumeSpecName "kube-api-access-4sxmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:48:10.208614 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.208495 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01c4c94c-4c5a-4996-9302-90088304e4c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "01c4c94c-4c5a-4996-9302-90088304e4c4" (UID: "01c4c94c-4c5a-4996-9302-90088304e4c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:48:10.228653 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.228613 2574 generic.go:356] "Generic (PLEG): container finished" podID="01c4c94c-4c5a-4996-9302-90088304e4c4" containerID="8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b" exitCode=0 Jul 07 13:48:10.228864 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.228669 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" Jul 07 13:48:10.228864 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.228697 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" event={"ID":"01c4c94c-4c5a-4996-9302-90088304e4c4","Type":"ContainerDied","Data":"8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b"} Jul 07 13:48:10.228864 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.228754 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx" event={"ID":"01c4c94c-4c5a-4996-9302-90088304e4c4","Type":"ContainerDied","Data":"21e9f6c5d8d4533664ff0aaae7135fb94efe51ac753d854ca7fdaa98afc4ed7a"} Jul 07 13:48:10.228864 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.228783 2574 scope.go:117] "RemoveContainer" containerID="8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b" Jul 07 13:48:10.239140 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.239114 2574 scope.go:117] "RemoveContainer" containerID="8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b" Jul 07 13:48:10.239515 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:48:10.239493 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b\": container with ID starting with 8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b not found: ID does not exist" containerID="8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b" Jul 07 13:48:10.239591 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.239526 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b"} err="failed to get container status \"8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b\": rpc error: code = NotFound desc = could not find container \"8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b\": container with ID starting with 8c9118d4a51ed74b376da657fbdf003fe3a5f1708bff7907c1274eec4d87ea0b not found: ID does not exist" Jul 07 13:48:10.264540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.264483 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx"] Jul 07 13:48:10.278994 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.278964 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-fe52df-688c457b8-4dvvx"] Jul 07 13:48:10.307469 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.307434 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sxmt\" (UniqueName: \"kubernetes.io/projected/01c4c94c-4c5a-4996-9302-90088304e4c4-kube-api-access-4sxmt\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:48:10.307469 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.307463 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c4c94c-4c5a-4996-9302-90088304e4c4-openshift-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:48:10.307469 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:10.307473 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c4c94c-4c5a-4996-9302-90088304e4c4-maas-api-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:48:11.432705 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:11.432667 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01c4c94c-4c5a-4996-9302-90088304e4c4" path="/var/lib/kubelet/pods/01c4c94c-4c5a-4996-9302-90088304e4c4/volumes" Jul 07 13:48:20.559670 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.559622 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5"] Jul 07 13:48:20.560275 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.560114 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01c4c94c-4c5a-4996-9302-90088304e4c4" containerName="maas-api" Jul 07 13:48:20.560275 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.560134 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="01c4c94c-4c5a-4996-9302-90088304e4c4" containerName="maas-api" Jul 07 13:48:20.560275 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.560161 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="889e1d75-9a8a-4355-80f5-9f61cbc70738" containerName="maas-api" Jul 07 13:48:20.560275 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.560169 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="889e1d75-9a8a-4355-80f5-9f61cbc70738" containerName="maas-api" Jul 07 13:48:20.560275 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.560275 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="01c4c94c-4c5a-4996-9302-90088304e4c4" containerName="maas-api" Jul 07 13:48:20.560546 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.560288 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="889e1d75-9a8a-4355-80f5-9f61cbc70738" containerName="maas-api" Jul 07 13:48:20.564913 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.564888 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:20.567846 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.567821 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pxjh4\"" Jul 07 13:48:20.570175 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.570145 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-db91fe-serving-cert\"" Jul 07 13:48:20.579455 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.579424 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5"] Jul 07 13:48:20.599716 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.599675 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12723b23-d17d-42e0-bd92-5bac82e97424-openshift-service-ca\") pod \"maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:20.599904 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.599731 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12723b23-d17d-42e0-bd92-5bac82e97424-maas-api-tls\") pod \"maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:20.599904 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.599793 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skgsp\" (UniqueName: \"kubernetes.io/projected/12723b23-d17d-42e0-bd92-5bac82e97424-kube-api-access-skgsp\") pod \"maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:20.700887 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.700845 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skgsp\" (UniqueName: \"kubernetes.io/projected/12723b23-d17d-42e0-bd92-5bac82e97424-kube-api-access-skgsp\") pod \"maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:20.701088 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.700907 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12723b23-d17d-42e0-bd92-5bac82e97424-openshift-service-ca\") pod \"maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:20.701088 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.700947 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12723b23-d17d-42e0-bd92-5bac82e97424-maas-api-tls\") pod \"maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:20.701088 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:48:20.701038 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-db91fe-serving-cert: secret "maas-api-e2e-shared-a-db91fe-serving-cert" not found Jul 07 13:48:20.701218 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:48:20.701091 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12723b23-d17d-42e0-bd92-5bac82e97424-maas-api-tls podName:12723b23-d17d-42e0-bd92-5bac82e97424 nodeName:}" failed. No retries permitted until 2026-07-07 13:48:21.201076153 +0000 UTC m=+2096.255497789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/12723b23-d17d-42e0-bd92-5bac82e97424-maas-api-tls") pod "maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" (UID: "12723b23-d17d-42e0-bd92-5bac82e97424") : secret "maas-api-e2e-shared-a-db91fe-serving-cert" not found Jul 07 13:48:20.701691 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.701670 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12723b23-d17d-42e0-bd92-5bac82e97424-openshift-service-ca\") pod \"maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:20.720974 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:20.720935 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skgsp\" (UniqueName: \"kubernetes.io/projected/12723b23-d17d-42e0-bd92-5bac82e97424-kube-api-access-skgsp\") pod \"maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:21.204937 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:21.204884 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12723b23-d17d-42e0-bd92-5bac82e97424-maas-api-tls\") pod \"maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:21.207519 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:21.207495 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12723b23-d17d-42e0-bd92-5bac82e97424-maas-api-tls\") pod \"maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:21.476868 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:21.476769 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:21.630808 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:21.630774 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5"] Jul 07 13:48:21.632489 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:48:21.632452 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12723b23_d17d_42e0_bd92_5bac82e97424.slice/crio-9047152920c497e543d9095b3c0bb35cd952a2028be5b91c2c1a4888f8d301d7 WatchSource:0}: Error finding container 9047152920c497e543d9095b3c0bb35cd952a2028be5b91c2c1a4888f8d301d7: Status 404 returned error can't find the container with id 9047152920c497e543d9095b3c0bb35cd952a2028be5b91c2c1a4888f8d301d7 Jul 07 13:48:22.300016 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:22.299975 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" event={"ID":"12723b23-d17d-42e0-bd92-5bac82e97424","Type":"ContainerStarted","Data":"6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f"} Jul 07 13:48:22.300279 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:22.300045 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:22.300279 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:22.300068 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" event={"ID":"12723b23-d17d-42e0-bd92-5bac82e97424","Type":"ContainerStarted","Data":"9047152920c497e543d9095b3c0bb35cd952a2028be5b91c2c1a4888f8d301d7"} Jul 07 13:48:22.394687 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:22.394567 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" podStartSLOduration=1.95264579 podStartE2EDuration="2.394547587s" totalImagesPullingTime="441.901797ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:48:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:48:22.390446553 +0000 UTC m=+2097.444868223" watchObservedRunningTime="2026-07-07 13:48:22.394547587 +0000 UTC m=+2097.448969260" Jul 07 13:48:28.312898 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:28.312866 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:48:28.857292 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:28.857237 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np"] Jul 07 13:48:28.862427 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:28.862398 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:28.872139 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:28.872101 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-2170c8-serving-cert\"" Jul 07 13:48:28.900149 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:28.900115 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np"] Jul 07 13:48:28.979478 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:28.979436 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:28.979478 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:28.979475 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47xcm\" (UniqueName: \"kubernetes.io/projected/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-kube-api-access-47xcm\") pod \"maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:28.979728 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:28.979612 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-maas-api-tls\") pod \"maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:29.080859 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:29.080822 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-maas-api-tls\") pod \"maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:29.081046 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:29.080873 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:29.081046 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:29.081007 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47xcm\" (UniqueName: \"kubernetes.io/projected/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-kube-api-access-47xcm\") pod \"maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:29.081512 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:29.081491 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:29.083902 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:29.083867 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-maas-api-tls\") pod \"maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:29.091777 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:29.091738 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47xcm\" (UniqueName: \"kubernetes.io/projected/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-kube-api-access-47xcm\") pod \"maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:29.174342 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:29.174208 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:29.324500 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:29.324466 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np"] Jul 07 13:48:29.326635 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:48:29.326604 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a0ea4a0_3a79_4af3_96ff_b4a2d5671553.slice/crio-eb5382979d96cbc28be88d7c8da65d7e82e62bf259a94b843c73979f9a9568bb WatchSource:0}: Error finding container eb5382979d96cbc28be88d7c8da65d7e82e62bf259a94b843c73979f9a9568bb: Status 404 returned error can't find the container with id eb5382979d96cbc28be88d7c8da65d7e82e62bf259a94b843c73979f9a9568bb Jul 07 13:48:29.340970 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:29.340929 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" event={"ID":"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553","Type":"ContainerStarted","Data":"eb5382979d96cbc28be88d7c8da65d7e82e62bf259a94b843c73979f9a9568bb"} Jul 07 13:48:30.348121 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:30.348077 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" event={"ID":"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553","Type":"ContainerStarted","Data":"33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94"} Jul 07 13:48:30.348613 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:30.348178 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:48:30.384334 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:30.384236 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" podStartSLOduration=1.867207338 podStartE2EDuration="2.384216256s" totalImagesPullingTime="517.008918ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:48:30.381958361 +0000 UTC m=+2105.436380020" watchObservedRunningTime="2026-07-07 13:48:30.384216256 +0000 UTC m=+2105.438637913" Jul 07 13:48:36.359979 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:48:36.359945 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:54:05.562240 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.562198 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5"] Jul 07 13:54:05.562733 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.562447 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" podUID="12723b23-d17d-42e0-bd92-5bac82e97424" containerName="maas-api" containerID="cri-o://6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f" gracePeriod=30 Jul 07 13:54:05.809591 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.809561 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:54:05.889758 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.889659 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12723b23-d17d-42e0-bd92-5bac82e97424-openshift-service-ca\") pod \"12723b23-d17d-42e0-bd92-5bac82e97424\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " Jul 07 13:54:05.889758 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.889716 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12723b23-d17d-42e0-bd92-5bac82e97424-maas-api-tls\") pod \"12723b23-d17d-42e0-bd92-5bac82e97424\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " Jul 07 13:54:05.889758 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.889751 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skgsp\" (UniqueName: \"kubernetes.io/projected/12723b23-d17d-42e0-bd92-5bac82e97424-kube-api-access-skgsp\") pod \"12723b23-d17d-42e0-bd92-5bac82e97424\" (UID: \"12723b23-d17d-42e0-bd92-5bac82e97424\") " Jul 07 13:54:05.890069 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.890046 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12723b23-d17d-42e0-bd92-5bac82e97424-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "12723b23-d17d-42e0-bd92-5bac82e97424" (UID: "12723b23-d17d-42e0-bd92-5bac82e97424"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:54:05.891968 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.891935 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12723b23-d17d-42e0-bd92-5bac82e97424-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "12723b23-d17d-42e0-bd92-5bac82e97424" (UID: "12723b23-d17d-42e0-bd92-5bac82e97424"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:54:05.891968 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.891938 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12723b23-d17d-42e0-bd92-5bac82e97424-kube-api-access-skgsp" (OuterVolumeSpecName: "kube-api-access-skgsp") pod "12723b23-d17d-42e0-bd92-5bac82e97424" (UID: "12723b23-d17d-42e0-bd92-5bac82e97424"). InnerVolumeSpecName "kube-api-access-skgsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:54:05.991281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.991205 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12723b23-d17d-42e0-bd92-5bac82e97424-maas-api-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:54:05.991281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.991240 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skgsp\" (UniqueName: \"kubernetes.io/projected/12723b23-d17d-42e0-bd92-5bac82e97424-kube-api-access-skgsp\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:54:05.991281 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:05.991282 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12723b23-d17d-42e0-bd92-5bac82e97424-openshift-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:54:06.202389 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:06.202291 2574 generic.go:356] "Generic (PLEG): container finished" podID="12723b23-d17d-42e0-bd92-5bac82e97424" containerID="6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f" exitCode=0 Jul 07 13:54:06.202389 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:06.202338 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" Jul 07 13:54:06.202389 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:06.202376 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" event={"ID":"12723b23-d17d-42e0-bd92-5bac82e97424","Type":"ContainerDied","Data":"6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f"} Jul 07 13:54:06.202609 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:06.202433 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5" event={"ID":"12723b23-d17d-42e0-bd92-5bac82e97424","Type":"ContainerDied","Data":"9047152920c497e543d9095b3c0bb35cd952a2028be5b91c2c1a4888f8d301d7"} Jul 07 13:54:06.202609 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:06.202456 2574 scope.go:117] "RemoveContainer" containerID="6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f" Jul 07 13:54:06.212883 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:06.212864 2574 scope.go:117] "RemoveContainer" containerID="6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f" Jul 07 13:54:06.213163 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:54:06.213141 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f\": container with ID starting with 6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f not found: ID does not exist" containerID="6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f" Jul 07 13:54:06.213214 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:06.213174 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f"} err="failed to get container status \"6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f\": rpc error: code = NotFound desc = could not find container \"6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f\": container with ID starting with 6175c7d899d8a0157e1b8c9589199e1c34167720a577a69d1142e9217db89f6f not found: ID does not exist" Jul 07 13:54:06.242968 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:06.242929 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5"] Jul 07 13:54:06.250676 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:06.250641 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-db91fe-7664cd57f5-f72f5"] Jul 07 13:54:07.437988 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:07.437942 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12723b23-d17d-42e0-bd92-5bac82e97424" path="/var/lib/kubelet/pods/12723b23-d17d-42e0-bd92-5bac82e97424/volumes" Jul 07 13:54:49.114445 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.114407 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np"] Jul 07 13:54:49.114946 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.114681 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" podUID="2a0ea4a0-3a79-4af3-96ff-b4a2d5671553" containerName="maas-api" containerID="cri-o://33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94" gracePeriod=30 Jul 07 13:54:49.367831 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.367800 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:54:49.377762 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.377730 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-maas-api-tls\") pod \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " Jul 07 13:54:49.377902 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.377774 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-openshift-service-ca\") pod \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " Jul 07 13:54:49.378134 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.378110 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2a0ea4a0-3a79-4af3-96ff-b4a2d5671553" (UID: "2a0ea4a0-3a79-4af3-96ff-b4a2d5671553"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:54:49.379956 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.379927 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2a0ea4a0-3a79-4af3-96ff-b4a2d5671553" (UID: "2a0ea4a0-3a79-4af3-96ff-b4a2d5671553"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:54:49.439360 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.439317 2574 generic.go:356] "Generic (PLEG): container finished" podID="2a0ea4a0-3a79-4af3-96ff-b4a2d5671553" containerID="33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94" exitCode=0 Jul 07 13:54:49.439360 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.439364 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" Jul 07 13:54:49.439617 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.439410 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" event={"ID":"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553","Type":"ContainerDied","Data":"33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94"} Jul 07 13:54:49.439617 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.439464 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np" event={"ID":"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553","Type":"ContainerDied","Data":"eb5382979d96cbc28be88d7c8da65d7e82e62bf259a94b843c73979f9a9568bb"} Jul 07 13:54:49.439617 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.439488 2574 scope.go:117] "RemoveContainer" containerID="33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94" Jul 07 13:54:49.448922 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.448893 2574 scope.go:117] "RemoveContainer" containerID="33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94" Jul 07 13:54:49.449293 ip-10-0-143-95 kubenswrapper[2574]: E0707 13:54:49.449267 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94\": container with ID starting with 33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94 not found: ID does not exist" containerID="33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94" Jul 07 13:54:49.449390 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.449308 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94"} err="failed to get container status \"33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94\": rpc error: code = NotFound desc = could not find container \"33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94\": container with ID starting with 33ced64d3e0b7701a150defc2a4b76a6920d6986b6c842d92583ee0d51035c94 not found: ID does not exist" Jul 07 13:54:49.478888 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.478854 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47xcm\" (UniqueName: \"kubernetes.io/projected/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-kube-api-access-47xcm\") pod \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\" (UID: \"2a0ea4a0-3a79-4af3-96ff-b4a2d5671553\") " Jul 07 13:54:49.479070 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.479036 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-maas-api-tls\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:54:49.479070 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.479046 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-openshift-service-ca\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:54:49.480968 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.480940 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-kube-api-access-47xcm" (OuterVolumeSpecName: "kube-api-access-47xcm") pod "2a0ea4a0-3a79-4af3-96ff-b4a2d5671553" (UID: "2a0ea4a0-3a79-4af3-96ff-b4a2d5671553"). InnerVolumeSpecName "kube-api-access-47xcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:54:49.579822 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.579780 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47xcm\" (UniqueName: \"kubernetes.io/projected/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553-kube-api-access-47xcm\") on node \"ip-10-0-143-95.ec2.internal\" DevicePath \"\"" Jul 07 13:54:49.762622 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.762581 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np"] Jul 07 13:54:49.766290 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:49.766262 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2170c8-74d7c7ff86-bt5np"] Jul 07 13:54:51.432522 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:54:51.432485 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a0ea4a0-3a79-4af3-96ff-b4a2d5671553" path="/var/lib/kubelet/pods/2a0ea4a0-3a79-4af3-96ff-b4a2d5671553/volumes" Jul 07 13:55:01.272773 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:01.272742 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723850-7zgzt_64dba76a-9892-4158-910a-3820c05f84e5/cleanup/0.log" Jul 07 13:55:01.384574 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:01.384539 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723865-hsvwj_beb2195c-d976-4a7d-bb5c-7524c64de3da/cleanup/0.log" Jul 07 13:55:01.876242 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:01.876211 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-56jhp_dc832fe3-7c94-4645-b49c-a99bd87a3e11/copy-manifests/0.log" Jul 07 13:55:01.885397 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:01.885368 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-56jhp_dc832fe3-7c94-4645-b49c-a99bd87a3e11/manager/0.log" Jul 07 13:55:03.086801 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.086756 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb_f457658d-bdeb-4d30-a224-6d995404d939/util/0.log" Jul 07 13:55:03.099741 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.099712 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb_f457658d-bdeb-4d30-a224-6d995404d939/pull/0.log" Jul 07 13:55:03.116688 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.116645 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb_f457658d-bdeb-4d30-a224-6d995404d939/extract/0.log" Jul 07 13:55:03.232744 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.232710 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx_dde178da-32ab-4bc8-b399-556e78e0ef42/util/0.log" Jul 07 13:55:03.243792 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.243766 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx_dde178da-32ab-4bc8-b399-556e78e0ef42/pull/0.log" Jul 07 13:55:03.256319 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.256290 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx_dde178da-32ab-4bc8-b399-556e78e0ef42/extract/0.log" Jul 07 13:55:03.368514 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.368432 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64_017bced3-b77f-407d-b471-00700fd3deac/util/0.log" Jul 07 13:55:03.376440 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.376410 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64_017bced3-b77f-407d-b471-00700fd3deac/pull/0.log" Jul 07 13:55:03.384800 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.384771 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64_017bced3-b77f-407d-b471-00700fd3deac/extract/0.log" Jul 07 13:55:03.501463 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.501424 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk_8500662d-2cd4-4a8c-a6ae-693736363282/extract/0.log" Jul 07 13:55:03.512056 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.512029 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk_8500662d-2cd4-4a8c-a6ae-693736363282/util/0.log" Jul 07 13:55:03.520633 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.520607 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk_8500662d-2cd4-4a8c-a6ae-693736363282/pull/0.log" Jul 07 13:55:03.785592 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:03.785553 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mnxbc_45159cff-f6eb-4e25-84ae-6a3999fa501c/manager/0.log" Jul 07 13:55:04.161772 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:04.161686 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xbpw4_84137efc-71b5-4674-9e3f-5c14a442a9f6/registry-server/0.log" Jul 07 13:55:04.302485 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:04.302450 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-42vhx_00c8bf29-1cd9-4de3-a19b-b55123a99c52/manager/0.log" Jul 07 13:55:04.549050 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:04.549012 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-tqf8j_8199209b-a00e-410e-9e8d-d6e2ac6230b3/manager/0.log" Jul 07 13:55:05.049447 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:05.049408 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-r4qsv_78b96415-a199-4f0d-99ee-24aa1a8f879f/discovery/0.log" Jul 07 13:55:05.281036 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:05.281001 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747f4757f-blw6t_b21a741e-f724-4457-957e-f8162defa8be/kube-auth-proxy/0.log" Jul 07 13:55:05.756888 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:05.756854 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58547f567f-h6ff4_6d6726a9-eb0f-45db-b9e9-5068cbda8413/router/0.log" Jul 07 13:55:06.220472 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:06.220444 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-pvdqb_d93924ae-d3cd-4c0b-97e6-263b0eeddc47/main/0.log" Jul 07 13:55:06.336363 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:06.336328 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-stchm_b42c1c4d-6519-48a0-9dc0-4feaeaa08a48/main/0.log" Jul 07 13:55:14.053651 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:14.053612 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tq86p_2bb5f676-7523-4cab-b7de-fd7cbd99581d/global-pull-secret-syncer/0.log" Jul 07 13:55:14.238795 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:14.238760 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zjrkx_ff29c277-a852-463a-8e9f-e066329a2095/konnectivity-agent/0.log" Jul 07 13:55:14.362699 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:14.362603 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-95.ec2.internal_18e3cb94201ad665b039cc13756723f0/haproxy/0.log" Jul 07 13:55:18.317851 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.317818 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb_f457658d-bdeb-4d30-a224-6d995404d939/extract/0.log" Jul 07 13:55:18.353560 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.353528 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb_f457658d-bdeb-4d30-a224-6d995404d939/util/0.log" Jul 07 13:55:18.389899 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.389864 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s6hvb_f457658d-bdeb-4d30-a224-6d995404d939/pull/0.log" Jul 07 13:55:18.429428 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.429394 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx_dde178da-32ab-4bc8-b399-556e78e0ef42/extract/0.log" Jul 07 13:55:18.452205 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.452169 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx_dde178da-32ab-4bc8-b399-556e78e0ef42/util/0.log" Jul 07 13:55:18.479460 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.479429 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cplhx_dde178da-32ab-4bc8-b399-556e78e0ef42/pull/0.log" Jul 07 13:55:18.507753 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.507720 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64_017bced3-b77f-407d-b471-00700fd3deac/extract/0.log" Jul 07 13:55:18.534110 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.534077 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64_017bced3-b77f-407d-b471-00700fd3deac/util/0.log" Jul 07 13:55:18.591726 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.591633 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732px64_017bced3-b77f-407d-b471-00700fd3deac/pull/0.log" Jul 07 13:55:18.677082 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.677053 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk_8500662d-2cd4-4a8c-a6ae-693736363282/extract/0.log" Jul 07 13:55:18.723743 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.723709 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk_8500662d-2cd4-4a8c-a6ae-693736363282/util/0.log" Jul 07 13:55:18.823599 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:18.823566 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qt5wk_8500662d-2cd4-4a8c-a6ae-693736363282/pull/0.log" Jul 07 13:55:19.371263 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:19.371214 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mnxbc_45159cff-f6eb-4e25-84ae-6a3999fa501c/manager/0.log" Jul 07 13:55:19.486067 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:19.486030 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xbpw4_84137efc-71b5-4674-9e3f-5c14a442a9f6/registry-server/0.log" Jul 07 13:55:19.651041 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:19.650944 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-42vhx_00c8bf29-1cd9-4de3-a19b-b55123a99c52/manager/0.log" Jul 07 13:55:19.774446 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:19.774395 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-tqf8j_8199209b-a00e-410e-9e8d-d6e2ac6230b3/manager/0.log" Jul 07 13:55:21.868719 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:21.868687 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mz4st_982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d/node-exporter/0.log" Jul 07 13:55:21.892311 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:21.892279 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mz4st_982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d/kube-rbac-proxy/0.log" Jul 07 13:55:21.915432 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:21.915403 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mz4st_982ce6f0-c8a6-4ac9-bd2e-b71cceb7e37d/init-textfile/0.log" Jul 07 13:55:22.402664 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:22.402627 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-chbmx_9acd5758-f64c-4851-8475-5b684fdc8d55/prometheus-operator-admission-webhook/0.log" Jul 07 13:55:23.624540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.624495 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7"] Jul 07 13:55:23.624952 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.624865 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a0ea4a0-3a79-4af3-96ff-b4a2d5671553" containerName="maas-api" Jul 07 13:55:23.624952 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.624879 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a0ea4a0-3a79-4af3-96ff-b4a2d5671553" containerName="maas-api" Jul 07 13:55:23.624952 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.624890 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12723b23-d17d-42e0-bd92-5bac82e97424" containerName="maas-api" Jul 07 13:55:23.624952 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.624896 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="12723b23-d17d-42e0-bd92-5bac82e97424" containerName="maas-api" Jul 07 13:55:23.624952 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.624953 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a0ea4a0-3a79-4af3-96ff-b4a2d5671553" containerName="maas-api" Jul 07 13:55:23.625119 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.624964 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="12723b23-d17d-42e0-bd92-5bac82e97424" containerName="maas-api" Jul 07 13:55:23.628156 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.628135 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.634004 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.633975 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g27ln\"/\"kube-root-ca.crt\"" Jul 07 13:55:23.634302 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.633985 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g27ln\"/\"default-dockercfg-vwth7\"" Jul 07 13:55:23.634547 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.634525 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g27ln\"/\"openshift-service-ca.crt\"" Jul 07 13:55:23.636621 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.636598 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7"] Jul 07 13:55:23.772428 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.772386 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vprt\" (UniqueName: \"kubernetes.io/projected/44bd37c4-976d-411f-b5c2-e29d141d83f4-kube-api-access-5vprt\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.772626 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.772458 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-sys\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.772626 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.772524 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-lib-modules\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.772738 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.772624 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-podres\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.772738 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.772661 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-proc\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.873103 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.873057 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-podres\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.873103 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.873105 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-proc\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.873403 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.873141 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vprt\" (UniqueName: \"kubernetes.io/projected/44bd37c4-976d-411f-b5c2-e29d141d83f4-kube-api-access-5vprt\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.873403 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.873179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-sys\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.873403 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.873181 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-podres\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.873403 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.873211 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-lib-modules\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.873403 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.873236 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-proc\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.873403 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.873301 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-sys\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.873403 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.873325 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/44bd37c4-976d-411f-b5c2-e29d141d83f4-lib-modules\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.884221 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.884146 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vprt\" (UniqueName: \"kubernetes.io/projected/44bd37c4-976d-411f-b5c2-e29d141d83f4-kube-api-access-5vprt\") pod \"perf-node-gather-daemonset-fv4m7\" (UID: \"44bd37c4-976d-411f-b5c2-e29d141d83f4\") " pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:23.939880 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:23.939839 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:24.098348 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:24.098318 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7"] Jul 07 13:55:24.100960 ip-10-0-143-95 kubenswrapper[2574]: W0707 13:55:24.100926 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod44bd37c4_976d_411f_b5c2_e29d141d83f4.slice/crio-41901c6f768054cdf75af52010eb0b54fade73c6450f7a0abdb22f1dbb2e3cea WatchSource:0}: Error finding container 41901c6f768054cdf75af52010eb0b54fade73c6450f7a0abdb22f1dbb2e3cea: Status 404 returned error can't find the container with id 41901c6f768054cdf75af52010eb0b54fade73c6450f7a0abdb22f1dbb2e3cea Jul 07 13:55:24.627530 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:24.627486 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" event={"ID":"44bd37c4-976d-411f-b5c2-e29d141d83f4","Type":"ContainerStarted","Data":"cb756711bf954fe8c640a2032cf4bd13309df242524815c73478112efe56af39"} Jul 07 13:55:24.627982 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:24.627554 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:24.627982 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:24.627578 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" event={"ID":"44bd37c4-976d-411f-b5c2-e29d141d83f4","Type":"ContainerStarted","Data":"41901c6f768054cdf75af52010eb0b54fade73c6450f7a0abdb22f1dbb2e3cea"} Jul 07 13:55:24.649183 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:24.649126 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" podStartSLOduration=1.649108058 podStartE2EDuration="1.649108058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:55:24.647426196 +0000 UTC m=+2519.701847880" watchObservedRunningTime="2026-07-07 13:55:24.649108058 +0000 UTC m=+2519.703529717" Jul 07 13:55:25.003832 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:25.003797 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-2jh4q_4efdac5c-6234-4954-9fe4-5b2d5dcb2303/download-server/0.log" Jul 07 13:55:26.683887 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:26.683852 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r6464_aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d/dns/0.log" Jul 07 13:55:26.712579 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:26.712545 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r6464_aae54f2e-3f86-4f77-b5d3-e8c5c875fb1d/kube-rbac-proxy/0.log" Jul 07 13:55:26.887471 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:26.887443 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tzl89_c9f3c64c-d89e-4583-955e-66d1c534336d/dns-node-resolver/0.log" Jul 07 13:55:27.489540 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:27.489508 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-txrsh_3829f252-2935-4883-9ce3-b3823a2c3572/node-ca/0.log" Jul 07 13:55:28.785204 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:28.785160 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-r4qsv_78b96415-a199-4f0d-99ee-24aa1a8f879f/discovery/0.log" Jul 07 13:55:28.895950 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:28.895918 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747f4757f-blw6t_b21a741e-f724-4457-957e-f8162defa8be/kube-auth-proxy/0.log" Jul 07 13:55:29.072270 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:29.072170 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58547f567f-h6ff4_6d6726a9-eb0f-45db-b9e9-5068cbda8413/router/0.log" Jul 07 13:55:29.797903 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:29.797866 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d4zmg_64b4c5e4-1a86-445f-9ae2-8fa8a1667ccf/serve-healthcheck-canary/0.log" Jul 07 13:55:30.532714 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:30.532681 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mccwm_e144ab3c-4492-409c-9ac1-06fad984629b/kube-rbac-proxy/0.log" Jul 07 13:55:30.557572 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:30.557542 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mccwm_e144ab3c-4492-409c-9ac1-06fad984629b/exporter/0.log" Jul 07 13:55:30.581437 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:30.581407 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mccwm_e144ab3c-4492-409c-9ac1-06fad984629b/extractor/0.log" Jul 07 13:55:30.643295 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:30.643237 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g27ln/perf-node-gather-daemonset-fv4m7" Jul 07 13:55:33.370712 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:33.370678 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723850-7zgzt_64dba76a-9892-4158-910a-3820c05f84e5/cleanup/0.log" Jul 07 13:55:33.399953 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:33.399925 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723865-hsvwj_beb2195c-d976-4a7d-bb5c-7524c64de3da/cleanup/0.log" Jul 07 13:55:33.765239 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:33.765209 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-56jhp_dc832fe3-7c94-4645-b49c-a99bd87a3e11/manager/0.log" Jul 07 13:55:33.830115 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:33.830023 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bdc7d5545-56jhp_dc832fe3-7c94-4645-b49c-a99bd87a3e11/copy-manifests/0.log" Jul 07 13:55:34.113044 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:34.112946 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-r4qsv_78b96415-a199-4f0d-99ee-24aa1a8f879f/discovery/0.log" Jul 07 13:55:34.173672 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:34.173641 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747f4757f-blw6t_b21a741e-f724-4457-957e-f8162defa8be/kube-auth-proxy/0.log" Jul 07 13:55:34.305086 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:34.305050 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58547f567f-h6ff4_6d6726a9-eb0f-45db-b9e9-5068cbda8413/router/0.log" Jul 07 13:55:35.721936 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:35.721901 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-vxv75_5be6b9dd-ce78-4adc-b14c-f9920f389b0b/openshift-lws-operator/0.log" Jul 07 13:55:35.815706 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:35.815665 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-r4qsv_78b96415-a199-4f0d-99ee-24aa1a8f879f/discovery/0.log" Jul 07 13:55:35.971156 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:35.971120 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747f4757f-blw6t_b21a741e-f724-4457-957e-f8162defa8be/kube-auth-proxy/0.log" Jul 07 13:55:36.151736 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:36.151698 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58547f567f-h6ff4_6d6726a9-eb0f-45db-b9e9-5068cbda8413/router/0.log" Jul 07 13:55:41.183104 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:41.183067 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-g4q8r_7f569c81-186b-46b5-a6a3-7d10df78af9f/kube-storage-version-migrator-operator/1.log" Jul 07 13:55:41.184728 ip-10-0-143-95 kubenswrapper[2574]: I0707 13:55:41.184702 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-g4q8r_7f569c81-186b-46b5-a6a3-7d10df78af9f/kube-storage-version-migrator-operator/0.log"