Aug 04 20:38:49.143035 ip-10-0-129-249 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Aug 04 20:38:49.143049 ip-10-0-129-249 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Aug 04 20:38:49.143058 ip-10-0-129-249 systemd[1]: kubelet.service: Failed with result 'resources'.
Aug 04 20:38:49.143356 ip-10-0-129-249 systemd[1]: Failed to start Kubernetes Kubelet.
Aug 04 20:38:59.375458 ip-10-0-129-249 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Aug 04 20:38:59.375478 ip-10-0-129-249 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 6aa174e4fe4a448d8f9240eb44841fe6 --
Aug 04 20:40:50.493630 ip-10-0-129-249 systemd[1]: Starting Kubernetes Kubelet...
Aug 04 20:40:50.853683 ip-10-0-129-249 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 20:40:50.853683 ip-10-0-129-249 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 04 20:40:50.853683 ip-10-0-129-249 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 20:40:50.853683 ip-10-0-129-249 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 04 20:40:50.853683 ip-10-0-129-249 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 20:40:50.854590 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.854521    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 04 20:40:50.857746 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857732    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 20:40:50.857746 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857746    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857749    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857753    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857757    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857760    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857763    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857766    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857769    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857771    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857774    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857776    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857779    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857782    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857785    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857788    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857790    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857793    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857795    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857798    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857800    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 20:40:50.857804 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857803    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857805    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857808    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857810    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857813    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857816    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857819    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857821    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857824    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857827    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857832    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857836    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857839    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857843    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857846    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857849    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857852    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857854    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857857    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 20:40:50.858304 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857859    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857862    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857864    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857867    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857869    2563 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857871    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857874    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857876    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857879    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857881    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857883    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857886    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857888    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857891    2563 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857893    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857897    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857900    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857902    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857905    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857907    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 20:40:50.858769 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857910    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857912    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857914    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857917    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857919    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857922    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857924    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857927    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857929    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857932    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857934    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857943    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857946    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857948    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857951    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857953    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857956    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857959    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857961    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857963    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 20:40:50.859243 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857966    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857968    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857970    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857973    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857975    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857977    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857980    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857983    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857986    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857988    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857991    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857993    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857996    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.857998    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858000    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858003    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858005    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858008    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858010    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858742    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 20:40:50.859719 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858749    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858752    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858755    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858758    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858761    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858764    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858766    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858769    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858772    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858774    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858777    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858780    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858783    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858785    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858788    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858790    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858793    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858795    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858797    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858800    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 20:40:50.860248 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858803    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858805    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858808    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858811    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858813    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858816    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858818    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858821    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858823    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858825    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858828    2563 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858830    2563 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858832    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858835    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858839    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858842    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858844    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858847    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858849    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858852    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 20:40:50.860742 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858854    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858856    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858859    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858862    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858864    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858867    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858869    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858873    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858877    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858879    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858882    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858884    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858887    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858890    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858893    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858895    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858897    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858901    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858903    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858905    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 20:40:50.861213 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858908    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858910    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858912    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858915    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858917    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858920    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858922    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858924    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858927    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858929    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858932    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858935    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858937    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858939    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858942    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858945    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858947    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858950    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858952    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858955    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 20:40:50.861725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858957    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858960    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858962    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858965    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858967    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858970    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858972    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858976    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858979    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858982    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858985    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858987    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858990    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858993    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858996    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.858998    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859001    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859004    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859414    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 20:40:50.862212 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859421    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859425    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859428    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859432    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859434    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859437    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859440    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859442    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859445    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859448    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859450    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859453    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859455    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859457    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859459    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859462    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859465    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859468    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859470    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 20:40:50.862685 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859472    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859475    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859477    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859480    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859482    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859485    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859487    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859490    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859492    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859494    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859497    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859499    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859501    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859504    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859506    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859508    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859511    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859514    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859516    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859519    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 20:40:50.863182 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859521    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859523    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859526    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859528    2563 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859530    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859533    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859536    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859539    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859542    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859545    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859548    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859550    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859553    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859556    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859558    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859560    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859563    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859565    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859567    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 20:40:50.863680 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859570    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859572    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859574    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859577    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859581    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859584    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859587    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859590    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859593    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859595    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859597    2563 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859600    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859603    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859605    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859608    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859611    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859613    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859615    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859618    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859621    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 20:40:50.864138 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859623    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859626    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859628    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859630    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859633    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859635    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859637    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859640    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859642    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859644    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859646    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859649    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859652    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859655    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859658    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859660    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859663    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859665    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859667    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.859670    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859733    2563 flags.go:64] FLAG: --address="0.0.0.0"
Aug 04 20:40:50.864651 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859739    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859745    2563 flags.go:64] FLAG: --anonymous-auth="true"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859749    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859755    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859759    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859765    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859772    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859777    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859782    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859785    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859788    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859791    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859793    2563 flags.go:64] FLAG: --cgroup-root=""
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859796    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859799    2563 flags.go:64] FLAG: --client-ca-file=""
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859801    2563 flags.go:64] FLAG: --cloud-provider="external"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859804    2563 flags.go:64] FLAG: --cluster-dns="[]"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859812    2563 flags.go:64] FLAG: --cluster-domain=""
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859814    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859817    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859820    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859823    2563 flags.go:64] FLAG: --container-log-max-files="5"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859827    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 04 20:40:50.865145 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859829    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859833    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859835    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859838    2563 flags.go:64] FLAG: --contention-profiling="false"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859841    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859843    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859846    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859849    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859852    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859855    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859858    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859861    2563 flags.go:64] FLAG: --enable-load-reader="false"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859863    2563 flags.go:64] FLAG: --enable-server="true"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859870    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859886    2563 flags.go:64] FLAG: --event-burst="100"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859889    2563 flags.go:64] FLAG: --event-qps="50"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859892    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859895    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859898    2563 flags.go:64] FLAG: --eviction-hard=""
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859902    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859904    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859907    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859909    2563 flags.go:64] FLAG: --eviction-soft=""
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859912    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859915    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 04 20:40:50.865695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859917    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859920    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859922    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859925    2563 flags.go:64] FLAG: --fail-swap-on="true"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859928    2563 flags.go:64] FLAG: --feature-gates=""
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859931    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859934    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859936    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859939    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859942    2563 flags.go:64] FLAG: --healthz-port="10248"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859945    2563 flags.go:64] FLAG: --help="false"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859947    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-129-249.ec2.internal"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859950    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859952    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859955    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859958    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859961    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859964    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859966    2563 flags.go:64] FLAG: --image-service-endpoint=""
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859969    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859973    2563 flags.go:64] FLAG: --kube-api-burst="100"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859976    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859979    2563 flags.go:64] FLAG: --kube-api-qps="50"
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859982    2563 flags.go:64] FLAG: --kube-reserved=""
Aug 04 20:40:50.866580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859985    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859987    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859990    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859993    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859995    2563 flags.go:64] FLAG: --lock-file=""
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.859998    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860000    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860004    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860009    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860011    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860014    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860016    2563 flags.go:64] FLAG: --logging-format="text"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860019    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860022    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860025    2563 flags.go:64] FLAG: --manifest-url=""
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860027    2563 flags.go:64] FLAG: --manifest-url-header=""
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860032    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860034    2563 flags.go:64] FLAG: --max-open-files="1000000"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860038    2563 flags.go:64] FLAG: --max-pods="110"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860041    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860044    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860046    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860049    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860052    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860054    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 04 20:40:50.867587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860057    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860063    2563 flags.go:64] FLAG: --node-status-max-images="50"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860066    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860070    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860073    2563 flags.go:64] FLAG: --pod-cidr=""
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860076    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860081    2563 flags.go:64] FLAG: --pod-manifest-path=""
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860083    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860086    2563 flags.go:64] FLAG: --pods-per-core="0"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860089    2563 flags.go:64] FLAG: --port="10250"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860091    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860094    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c9957a77b1fccde5"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860097    2563 flags.go:64] FLAG: --qos-reserved=""
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860100    2563 flags.go:64] FLAG: --read-only-port="10255"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860102    2563 flags.go:64] FLAG: --register-node="true"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860105    2563 flags.go:64] FLAG: --register-with-taints=""
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860108    2563 flags.go:64] FLAG: --registry-burst="10"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860111    2563 flags.go:64] FLAG: --registry-qps="5"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860113    2563 flags.go:64] FLAG: --reserved-cpus=""
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860116    2563 flags.go:64] FLAG: --reserved-memory=""
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860119    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860122    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860125    2563 flags.go:64] FLAG: --rotate-certificates="false"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860127    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860130    2563 flags.go:64] FLAG: --runonce="false"
Aug 04 20:40:50.868637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860132    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860135    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860138    2563 flags.go:64] FLAG: --seccomp-default="false"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860141    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860143    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860146    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860148    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860151    2563 flags.go:64] FLAG: --storage-driver-password="root"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860154    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860156    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860160    2563 flags.go:64] FLAG: --storage-driver-user="root"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860164    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860166    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860170    2563 flags.go:64] FLAG: --system-cgroups=""
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860172    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860177    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860180    2563 flags.go:64] FLAG: --tls-cert-file=""
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860182    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860186    2563 flags.go:64] FLAG: --tls-min-version=""
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860188    2563 flags.go:64] FLAG: --tls-private-key-file=""
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860191    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860194    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860196    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860199    2563 flags.go:64] FLAG: --v="2"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860203    2563 flags.go:64] FLAG: --version="false"
Aug 04 20:40:50.869776 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860206    2563 flags.go:64] FLAG: --vmodule=""
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860209    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.860213    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860313    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860317    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860321    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860325    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860329    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860332    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860334    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860337    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860340    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860342    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860345    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860348    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860351    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860354    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860358    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 20:40:50.870563 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860362    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860365    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860381    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860384    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860387    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860389    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860392    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860395    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860397    2563 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860400    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860402    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860404    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860407    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860409    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860411    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860414    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860416    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860418    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860421    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860423    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 20:40:50.871288 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860425    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860428    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860430    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860432    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860435    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860437    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860440    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860442    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860445    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860447    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860449    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860454    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860457    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860460    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860462    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860465    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860468    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860470    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860473    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860475    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 20:40:50.871998 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860477    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860480    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860482    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860484    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860487    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860489    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860492    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860494    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860496    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860499    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860501    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860503    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860506    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860508    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860510    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860513    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860515    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860518    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860520    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860522    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 20:40:50.872717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860525    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860527    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860529    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860533    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860537    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860540    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860543    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860545    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860547    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860550    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860553    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860555    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860557    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860560    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860562    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860564    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860567    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860569    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860572    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860574    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 20:40:50.873398 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860576    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860579    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860581    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.860583    2563 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.861355    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.868305    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.868322    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.868346    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.868354    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868459    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868467    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868472    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868477    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868481    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 20:40:50.874100 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868485    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868489    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868494    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868498    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868504    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868510    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868514    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868518    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868523    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868528    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868532    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868536    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868542    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868548    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868552    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868557    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868561    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868566    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868571    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 20:40:50.874717 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868575    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868579    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868583    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868587    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868592    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868605    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868610    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868614    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868618    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868622    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868625    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868630    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868634    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868637    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868641    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868645    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868648    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868652    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868656    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868660    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 20:40:50.875336 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868664    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868667    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868671    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868675    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868679    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868683    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868687    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868691    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868695    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868699    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868703    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868709    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868713    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868717    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868722    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868726    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868730    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868735    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868746    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 20:40:50.876041 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868750    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868754    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868758    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868762    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868766    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868770    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868773    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868777    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868781    2563 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868785    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868789    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868793    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868797    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868801    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868805    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868809    2563 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868813    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868817    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868821    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868824    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868828    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 20:40:50.876550 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868832    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868836    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868840    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868844    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868848    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868852    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868856    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868860    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868865    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868869    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868873    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868884    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868888    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868892    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.868896    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 20:40:50.877062 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.868904    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869121    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869129    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869133    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869137    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869142    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869145    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869149    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869153    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869157    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869161    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869165    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869169    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869174    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869179    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869183    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869187    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869194    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869200    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869204    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 20:40:50.877452 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869208    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869212    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869216    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869220    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869224    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869228    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869232    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869236    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869240    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869252    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869256    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869260    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869264    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869268    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869272    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869276    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869280    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869284    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869287    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869291    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 20:40:50.877925 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869295    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869299    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869303    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869307    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869310    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869314    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869318    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869322    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869326    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869330    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869334    2563 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869338    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869342    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869346    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869350    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869354    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869357    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869361    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869365    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869393    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 20:40:50.878506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869397    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869401    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869412    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869417    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869422    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869426    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869429    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869433    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869437    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869442    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869446    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869450    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869454    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869458    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869461    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869466    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869470    2563 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869474    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869478    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869482    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 20:40:50.878982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869488    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869495    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869500    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869505    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869509    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869513    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869517    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869521    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869525    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869529    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869533    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869537    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869541    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869545    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869549    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869563    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869567    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869571    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869575    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 20:40:50.879499 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:50.869579    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 20:40:50.879959 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.869586    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 20:40:50.879959 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.869788    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 04 20:40:50.879959 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.872276    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 04 20:40:50.879959 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.876988    2563 server.go:1019] "Starting client certificate rotation"
Aug 04 20:40:50.879959 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.877088    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 20:40:50.879959 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.877783    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 20:40:50.879959 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.879247    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 20:40:50.880408 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.880394    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 20:40:50.891835 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.891814    2563 log.go:25] "Validated CRI v1 runtime API"
Aug 04 20:40:50.895920 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.895904    2563 log.go:25] "Validated CRI v1 image API"
Aug 04 20:40:50.897832 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.897817    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 04 20:40:50.900472 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.900446    2563 fs.go:135] Filesystem UUIDs: map[5af9d9b8-aaeb-4dd1-91ed-d8ab5c967447:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7cfb75cf-ad2b-42f0-a365-35de5207609c:/dev/nvme0n1p4]
Aug 04 20:40:50.900556 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.900470    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 04 20:40:50.905932 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.905824    2563 manager.go:216] Machine: {Timestamp:2026-08-04 20:40:50.904550773 +0000 UTC m=+0.303113093 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3163765 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21de1d67f0d45cd5c5fc1909c67e8d SystemUUID:ec21de1d-67f0-d45c-d5c5-fc1909c67e8d BootID:6aa174e4-fe4a-448d-8f92-40eb44841fe6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:aa:46:de:20:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:aa:46:de:20:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:79:64:50:3c:50 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 04 20:40:50.905932 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.905924    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 04 20:40:50.906083 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.906019    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 04 20:40:50.907634 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.907618    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 20:40:50.908554 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.908522    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 04 20:40:50.908689 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.908557    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-249.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 04 20:40:50.909777 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.909766    2563 topology_manager.go:138] "Creating topology manager with none policy"
Aug 04 20:40:50.909814 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.909781    2563 container_manager_linux.go:309] "Creating device plugin manager"
Aug 04 20:40:50.909814 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.909788    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 20:40:50.910478 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.910468    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 20:40:50.911162 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.911153    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 04 20:40:50.914457 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.914446    2563 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 20:40:50.914574 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.914564    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 04 20:40:50.914604 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.914597    2563 kubelet.go:486] "Attempting to sync node with API server"
Aug 04 20:40:50.914636 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.914609    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 04 20:40:50.914636 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.914623    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 04 20:40:50.914636 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.914631    2563 kubelet.go:398] "Adding apiserver pod source"
Aug 04 20:40:50.914741 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.914639    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 04 20:40:50.914959 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.914949    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 20:40:50.914991 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.914966    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 20:40:50.917198 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.917182    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 04 20:40:50.919037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.919015    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 04 20:40:50.919127 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.919058    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 04 20:40:50.920103 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920083    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 04 20:40:50.920185 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920112    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 04 20:40:50.920227 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920204    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 04 20:40:50.920227 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920223    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 04 20:40:50.920284 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920233    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 04 20:40:50.920284 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920244    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 04 20:40:50.920284 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920255    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 04 20:40:50.920284 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920262    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 04 20:40:50.920406 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920305    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 04 20:40:50.920406 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920318    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 04 20:40:50.920406 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920329    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 04 20:40:50.920406 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.920335    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 04 20:40:50.921241 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.921224    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 04 20:40:50.921289 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.921260    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 04 20:40:50.924243 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.924230    2563 server.go:1268] "Started kubelet"
Aug 04 20:40:50.924346 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.924321    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 04 20:40:50.924346 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.924312    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 04 20:40:50.924458 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.924415    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 04 20:40:50.924992 ip-10-0-129-249 systemd[1]: Started Kubernetes Kubelet.
Aug 04 20:40:50.925951 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.925851    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 04 20:40:50.926425 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.926412    2563 server.go:310] "Adding debug handlers to kubelet server"
Aug 04 20:40:50.927250 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.927222    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-249.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 04 20:40:50.927341 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.927303    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-249.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 04 20:40:50.927407 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.927314    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 04 20:40:50.929812 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.929796    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 04 20:40:50.931156 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.931138    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 04 20:40:50.931156 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.931148    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 04 20:40:50.931633 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.931285    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:50.931808 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.931781    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 04 20:40:50.932193 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.931811    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 04 20:40:50.932268 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.932196    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 04 20:40:50.932268 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.931925    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 04 20:40:50.932268 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.932214    2563 factory.go:55] Registering systemd factory
Aug 04 20:40:50.932268 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.932223    2563 factory.go:223] Registration of the systemd container factory successfully
Aug 04 20:40:50.932434 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.932308    2563 reconstruct.go:97] "Volume reconstruction finished"
Aug 04 20:40:50.932434 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.932318    2563 reconciler.go:29] "Reconciler: start to sync state"
Aug 04 20:40:50.932496 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.932457    2563 factory.go:153] Registering CRI-O factory
Aug 04 20:40:50.932496 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.932471    2563 factory.go:223] Registration of the crio container factory successfully
Aug 04 20:40:50.932496 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.932493    2563 factory.go:103] Registering Raw factory
Aug 04 20:40:50.932632 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.932506    2563 manager.go:1196] Started watching for new ooms in manager
Aug 04 20:40:50.933256 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.933240    2563 manager.go:349] Starting recovery of all containers
Aug 04 20:40:50.933750 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.932723    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-249.ec2.internal.18c8b4ce6c36f1ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-249.ec2.internal,UID:ip-10-0-129-249.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-249.ec2.internal,},FirstTimestamp:2026-08-04 20:40:50.924204494 +0000 UTC m=+0.322766812,LastTimestamp:2026-08-04 20:40:50.924204494 +0000 UTC m=+0.322766812,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-249.ec2.internal,}"
Aug 04 20:40:50.938433 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.937994    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 04 20:40:50.938433 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.938343    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-249.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 04 20:40:50.942645 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.942620    2563 manager.go:354] Recovery completed
Aug 04 20:40:50.944343 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.943390    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 04 20:40:50.948725 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.948629    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 20:40:50.950404 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.950386    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 20:40:50.950451 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.950417    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 20:40:50.950451 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.950427    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientPID"
Aug 04 20:40:50.950884 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.950871    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 04 20:40:50.950884 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.950882    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 04 20:40:50.951006 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.950896    2563 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 20:40:50.951006 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.950926    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 20:40:50.951309 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.951294    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cg56l"
Aug 04 20:40:50.951632 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.951617    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 20:40:50.951708 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.951643    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 20:40:50.951708 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.951656    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientPID"
Aug 04 20:40:50.952023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.952010    2563 policy_none.go:49] "None policy: Start"
Aug 04 20:40:50.952059 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.952028    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 04 20:40:50.952059 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.952038    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 04 20:40:50.952748 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.952670    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-249.ec2.internal.18c8b4ce6dc6b7f5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-249.ec2.internal,UID:ip-10-0-129-249.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-249.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-249.ec2.internal,},FirstTimestamp:2026-08-04 20:40:50.950404085 +0000 UTC m=+0.348966407,LastTimestamp:2026-08-04 20:40:50.950404085 +0000 UTC m=+0.348966407,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-249.ec2.internal,}"
Aug 04 20:40:50.952926 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.952906    2563 policy_none.go:47] "Start"
Aug 04 20:40:50.960449 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.960430    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cg56l"
Aug 04 20:40:50.963171 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.963117    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-249.ec2.internal.18c8b4ce6dc6fe09  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-249.ec2.internal,UID:ip-10-0-129-249.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-249.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-249.ec2.internal,},FirstTimestamp:2026-08-04 20:40:50.950422025 +0000 UTC m=+0.348984344,LastTimestamp:2026-08-04 20:40:50.950422025 +0000 UTC m=+0.348984344,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-249.ec2.internal,}"
Aug 04 20:40:50.995054 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.995042    2563 manager.go:337] "Starting Device Plugin manager"
Aug 04 20:40:50.995142 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.995069    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 04 20:40:50.995142 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.995079    2563 server.go:87] "Starting device plugin registration server"
Aug 04 20:40:50.995304 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.995291    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 04 20:40:50.995356 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.995305    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 04 20:40:50.995437 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.995406    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 04 20:40:50.995506 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.995492    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 04 20:40:50.995554 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:50.995507    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 04 20:40:50.996103 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.996084    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 04 20:40:50.996164 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:50.996136    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:51.081659 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.081644    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 04 20:40:51.082676 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.081673    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 04 20:40:51.082676 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.081695    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 04 20:40:51.082676 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.081725    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 04 20:40:51.086992 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.086975    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 20:40:51.096346 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.096328    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 20:40:51.097017 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.097003    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 20:40:51.097090 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.097028    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 20:40:51.097090 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.097037    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientPID"
Aug 04 20:40:51.097090 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.097056    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.105049 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.105010    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.105049 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.105036    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-249.ec2.internal\": node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:51.124936 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.124917    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:51.181847 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.181811    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-249.ec2.internal"]
Aug 04 20:40:51.181928 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.181877    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 20:40:51.182586 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.182574    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 20:40:51.182652 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.182609    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 20:40:51.182652 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.182619    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientPID"
Aug 04 20:40:51.183606 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.183595    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 20:40:51.183733 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.183719    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.183816 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.183750    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 20:40:51.184218 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.184202    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 20:40:51.184218 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.184218    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 20:40:51.184328 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.184228    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 20:40:51.184328 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.184233    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 20:40:51.184328 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.184242    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientPID"
Aug 04 20:40:51.184328 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.184243    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientPID"
Aug 04 20:40:51.185276 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.185264    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.185324 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.185288    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 20:40:51.185888 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.185871    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 20:40:51.185958 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.185903    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 20:40:51.185958 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.185917    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeHasSufficientPID"
Aug 04 20:40:51.211875 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.211859    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-249.ec2.internal\" not found" node="ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.216044 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.216031    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-249.ec2.internal\" not found" node="ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.224970 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.224959    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:51.235298 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.235285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3d8aaa7b15afe68782f913fe00a30060-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal\" (UID: \"3d8aaa7b15afe68782f913fe00a30060\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.235357 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.235305    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d8aaa7b15afe68782f913fe00a30060-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal\" (UID: \"3d8aaa7b15afe68782f913fe00a30060\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.235357 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.235321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f3d43927280795fd50a92f8253826fd4-config\") pod \"kube-apiserver-proxy-ip-10-0-129-249.ec2.internal\" (UID: \"f3d43927280795fd50a92f8253826fd4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.325269 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.325254    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:51.335614 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.335598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3d8aaa7b15afe68782f913fe00a30060-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal\" (UID: \"3d8aaa7b15afe68782f913fe00a30060\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.335671 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.335661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d8aaa7b15afe68782f913fe00a30060-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal\" (UID: \"3d8aaa7b15afe68782f913fe00a30060\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.335705 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.335680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d8aaa7b15afe68782f913fe00a30060-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal\" (UID: \"3d8aaa7b15afe68782f913fe00a30060\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.335705 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.335682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3d8aaa7b15afe68782f913fe00a30060-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal\" (UID: \"3d8aaa7b15afe68782f913fe00a30060\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.335766 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.335707    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f3d43927280795fd50a92f8253826fd4-config\") pod \"kube-apiserver-proxy-ip-10-0-129-249.ec2.internal\" (UID: \"f3d43927280795fd50a92f8253826fd4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.335766 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.335730    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f3d43927280795fd50a92f8253826fd4-config\") pod \"kube-apiserver-proxy-ip-10-0-129-249.ec2.internal\" (UID: \"f3d43927280795fd50a92f8253826fd4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.426185 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.426143    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:51.514586 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.514571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.518601 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.518441    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:51.526850 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.526833    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:51.627420 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.627391    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:51.727946 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.727909    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:51.828478 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.828462    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:51.876974 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.876959    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 04 20:40:51.877324 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.877061    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 20:40:51.928886 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:51.928869    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-249.ec2.internal\" not found"
Aug 04 20:40:51.964223 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.964195    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 20:35:53 +0000 UTC" deadline="2028-05-22 14:00:54.55639986 +0000 UTC"
Aug 04 20:40:51.964223 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.964219    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15761h20m2.592183938s"
Aug 04 20:40:51.975597 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:51.975579    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 20:40:52.031874 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:52.031856    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:52.046543 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:52.046524    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 20:40:52.048611 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:52.048596    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-249.ec2.internal"
Aug 04 20:40:52.061859 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:52.061844    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 20:40:52.066924 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:52.066902    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d8aaa7b15afe68782f913fe00a30060.slice/crio-0ddc89530446fb92892d0931ceed737a60aa2ef255dfbe25cbe5274ca731d34e WatchSource:0}: Error finding container 0ddc89530446fb92892d0931ceed737a60aa2ef255dfbe25cbe5274ca731d34e: Status 404 returned error can't find the container with id 0ddc89530446fb92892d0931ceed737a60aa2ef255dfbe25cbe5274ca731d34e
Aug 04 20:40:52.070632 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:52.070620    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 20:40:52.085663 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:52.085626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal" event={"ID":"3d8aaa7b15afe68782f913fe00a30060","Type":"ContainerStarted","Data":"0ddc89530446fb92892d0931ceed737a60aa2ef255dfbe25cbe5274ca731d34e"}
Aug 04 20:40:52.153507 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:52.153487    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 20:40:52.226436 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:52.226418    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d43927280795fd50a92f8253826fd4.slice/crio-87802b1dd2fa2aa0cd5fff06e117e183eb0180cc6dd123e0ebcca74cbb2d3688 WatchSource:0}: Error finding container 87802b1dd2fa2aa0cd5fff06e117e183eb0180cc6dd123e0ebcca74cbb2d3688: Status 404 returned error can't find the container with id 87802b1dd2fa2aa0cd5fff06e117e183eb0180cc6dd123e0ebcca74cbb2d3688
Aug 04 20:40:52.475725 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:52.475678    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 20:40:55.070905 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.070833    2563 apiserver.go:52] "Watching apiserver"
Aug 04 20:40:55.082238 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.082215    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 04 20:40:55.083817 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.083792    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-rpwv5","openshift-multus/multus-mk5vg","openshift-multus/network-metrics-daemon-8wdv9","kube-system/global-pull-secret-syncer-62hnz","kube-system/konnectivity-agent-vh5t2","openshift-cluster-node-tuning-operator/tuned-89np2","openshift-image-registry/node-ca-w7fs7","openshift-network-diagnostics/network-check-target-bkp8v","openshift-network-operator/iptables-alerter-w5ftp","openshift-ovn-kubernetes/ovnkube-node-8rgcs","kube-system/kube-apiserver-proxy-ip-10-0-129-249.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7","openshift-dns/node-resolver-kblfw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal"]
Aug 04 20:40:55.086341 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.085746    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w7fs7"
Aug 04 20:40:55.087796 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.086920    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kblfw"
Aug 04 20:40:55.087796 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.087264    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 04 20:40:55.088306 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.088107    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:40:55.088306 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.088159    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:40:55.089556 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.089376    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:55.089556 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.089436    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:40:55.091216 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.090647    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:40:55.091907 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.091673    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hwk4g\""
Aug 04 20:40:55.091907 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.091719    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 04 20:40:55.091907 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.091791    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 04 20:40:55.091907 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.091678    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 04 20:40:55.092144 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.091952    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l8xhf\""
Aug 04 20:40:55.092144 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.091980    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 04 20:40:55.092144 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.092124    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.092285 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.092158    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 04 20:40:55.093245 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.093226    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w5ftp"
Aug 04 20:40:55.095691 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.094878    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.097212 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.096643    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 04 20:40:55.097212 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.096857    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 04 20:40:55.097212 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.096861    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5dbpd\""
Aug 04 20:40:55.097212 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.096877    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 04 20:40:55.097212 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.097002    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-khxfb\""
Aug 04 20:40:55.097212 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.097034    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 04 20:40:55.097212 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.097166    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.097825 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.097807    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 04 20:40:55.097910 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.097841    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 04 20:40:55.098054 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.098038    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xv4lj\""
Aug 04 20:40:55.098125 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.098041    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 04 20:40:55.098125 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.098103    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 04 20:40:55.098458 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.098226    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 04 20:40:55.098458 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.098297    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 04 20:40:55.098599 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.098510    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9zz4j\""
Aug 04 20:40:55.098599 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.098589    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 04 20:40:55.098691 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.098601    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 04 20:40:55.099831 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.099339    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.099935 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.099825    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwfc2\" (UniqueName: \"kubernetes.io/projected/d0178830-4009-4007-9829-a2c0aee48925-kube-api-access-bwfc2\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:40:55.099935 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.099891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f558bf91-5880-4917-9e1e-c78ae5493458-serviceca\") pod \"node-ca-w7fs7\" (UID: \"f558bf91-5880-4917-9e1e-c78ae5493458\") " pod="openshift-image-registry/node-ca-w7fs7"
Aug 04 20:40:55.099935 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.099916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62b003b3-9dd3-4f30-b0ac-c88b3003e065-tmp-dir\") pod \"node-resolver-kblfw\" (UID: \"62b003b3-9dd3-4f30-b0ac-c88b3003e065\") " pod="openshift-dns/node-resolver-kblfw"
Aug 04 20:40:55.100037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.099942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzzb6\" (UniqueName: \"kubernetes.io/projected/62b003b3-9dd3-4f30-b0ac-c88b3003e065-kube-api-access-wzzb6\") pod \"node-resolver-kblfw\" (UID: \"62b003b3-9dd3-4f30-b0ac-c88b3003e065\") " pod="openshift-dns/node-resolver-kblfw"
Aug 04 20:40:55.100037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.099965    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:40:55.100037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.099988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f558bf91-5880-4917-9e1e-c78ae5493458-host\") pod \"node-ca-w7fs7\" (UID: \"f558bf91-5880-4917-9e1e-c78ae5493458\") " pod="openshift-image-registry/node-ca-w7fs7"
Aug 04 20:40:55.100037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.100010    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npcnj\" (UniqueName: \"kubernetes.io/projected/f558bf91-5880-4917-9e1e-c78ae5493458-kube-api-access-npcnj\") pod \"node-ca-w7fs7\" (UID: \"f558bf91-5880-4917-9e1e-c78ae5493458\") " pod="openshift-image-registry/node-ca-w7fs7"
Aug 04 20:40:55.100037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.100033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62b003b3-9dd3-4f30-b0ac-c88b3003e065-hosts-file\") pod \"node-resolver-kblfw\" (UID: \"62b003b3-9dd3-4f30-b0ac-c88b3003e065\") " pod="openshift-dns/node-resolver-kblfw"
Aug 04 20:40:55.100466 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.100447    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:40:55.100546 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.100517    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:40:55.101819 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.101802    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.105081 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.105064    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6dnst\""
Aug 04 20:40:55.105172 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.105092    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 04 20:40:55.105536 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.105518    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 04 20:40:55.105536 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.105531    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 04 20:40:55.105732 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.105715    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 04 20:40:55.105793 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.105766    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hvqnw\""
Aug 04 20:40:55.106198 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.106086    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 04 20:40:55.108105 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.108087    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 04 20:40:55.108190 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.108157    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 04 20:40:55.108289 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.108276    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 04 20:40:55.108436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.108418    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4fs62\""
Aug 04 20:40:55.108488 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.108445    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 04 20:40:55.108538 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.108496    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 20:40:55.108588 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.108095    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 04 20:40:55.143595 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.143579    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-95c2p"
Aug 04 20:40:55.159245 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.159225    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-95c2p"
Aug 04 20:40:55.188364 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.188346    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 04 20:40:55.200211 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-socket-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.200336 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-os-release\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.200336 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-run-multus-certs\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.200336 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200257    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-run-ovn\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.200336 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200279    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.200336 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200296    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a951e2c-81e0-4776-9f94-e4693a89d219-env-overrides\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.200535 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200338    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-dbus\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:55.200535 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200358    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-system-cni-dir\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.200535 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200396    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-cnibin\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.200535 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200456    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npcnj\" (UniqueName: \"kubernetes.io/projected/f558bf91-5880-4917-9e1e-c78ae5493458-kube-api-access-npcnj\") pod \"node-ca-w7fs7\" (UID: \"f558bf91-5880-4917-9e1e-c78ae5493458\") " pod="openshift-image-registry/node-ca-w7fs7"
Aug 04 20:40:55.200535 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwfc2\" (UniqueName: \"kubernetes.io/projected/d0178830-4009-4007-9829-a2c0aee48925-kube-api-access-bwfc2\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:40:55.200723 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200554    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-system-cni-dir\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.200723 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200572    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-cni-binary-copy\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.200723 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200603    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-var-lib-cni-multus\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.200723 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-kubelet\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.200723 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhstm\" (UniqueName: \"kubernetes.io/projected/4a951e2c-81e0-4776-9f94-e4693a89d219-kube-api-access-zhstm\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.200723 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-etc-selinux\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.200723 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:40:55.200723 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-kubelet-config\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:55.201008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200726    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-run\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.201008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8jwg\" (UniqueName: \"kubernetes.io/projected/2eea297c-3921-4073-a019-78634486f6dc-kube-api-access-l8jwg\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.201008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-cnibin\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.201008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-run-k8s-cni-cncf-io\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.201008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-systemd-units\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.201008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200823    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-cni-netd\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.201008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a951e2c-81e0-4776-9f94-e4693a89d219-ovn-node-metrics-cert\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.201008 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.200911    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:40:55.201008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-os-release\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.201008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-sysconfig\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.201008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.200988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-cni-bin\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.201064    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs podName:d0178830-4009-4007-9829-a2c0aee48925 nodeName:}" failed. No retries permitted until 2026-08-04 20:40:55.701037421 +0000 UTC m=+2.944656697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs") pod "network-metrics-daemon-8wdv9" (UID: "d0178830-4009-4007-9829-a2c0aee48925") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6g28\" (UniqueName: \"kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28\") pod \"network-check-target-bkp8v\" (UID: \"c1997346-3963-43df-b2f3-515df3356344\") " pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201111    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7876b090-ff49-42b7-aff9-93b9aa25ef12-agent-certs\") pod \"konnectivity-agent-vh5t2\" (UID: \"7876b090-ff49-42b7-aff9-93b9aa25ef12\") " pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a42c0953-c4c7-4aed-b344-9b63c27267bb-host-slash\") pod \"iptables-alerter-w5ftp\" (UID: \"a42c0953-c4c7-4aed-b344-9b63c27267bb\") " pod="openshift-network-operator/iptables-alerter-w5ftp"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2eea297c-3921-4073-a019-78634486f6dc-etc-tuned\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-var-lib-cni-bin\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-var-lib-kubelet\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201261    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-conf-dir\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201284    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-etc-kubernetes\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-sysctl-conf\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201359    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-hostroot\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201389    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-daemon-config\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.201431 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-node-log\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.201984 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-slash\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.201984 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-run-netns\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.201984 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a951e2c-81e0-4776-9f94-e4693a89d219-ovnkube-script-lib\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.201984 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:55.201984 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201687    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjzfd\" (UniqueName: \"kubernetes.io/projected/283ad954-19e6-44b0-8c87-bb257b1e4a97-kube-api-access-zjzfd\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.201984 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f558bf91-5880-4917-9e1e-c78ae5493458-serviceca\") pod \"node-ca-w7fs7\" (UID: \"f558bf91-5880-4917-9e1e-c78ae5493458\") " pod="openshift-image-registry/node-ca-w7fs7"
Aug 04 20:40:55.201984 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201877    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62b003b3-9dd3-4f30-b0ac-c88b3003e065-tmp-dir\") pod \"node-resolver-kblfw\" (UID: \"62b003b3-9dd3-4f30-b0ac-c88b3003e065\") " pod="openshift-dns/node-resolver-kblfw"
Aug 04 20:40:55.201984 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2eea297c-3921-4073-a019-78634486f6dc-tmp\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.202348 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.201993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5t24\" (UniqueName: \"kubernetes.io/projected/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-kube-api-access-k5t24\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.202348 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a951e2c-81e0-4776-9f94-e4693a89d219-ovnkube-config\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.202348 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a42c0953-c4c7-4aed-b344-9b63c27267bb-iptables-alerter-script\") pod \"iptables-alerter-w5ftp\" (UID: \"a42c0953-c4c7-4aed-b344-9b63c27267bb\") " pod="openshift-network-operator/iptables-alerter-w5ftp"
Aug 04 20:40:55.202348 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrsbg\" (UniqueName: \"kubernetes.io/projected/9b89dd0c-99b7-4dce-96c6-3f8e90482915-kube-api-access-wrsbg\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.202348 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202257    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-var-lib-kubelet\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.202348 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202290    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62b003b3-9dd3-4f30-b0ac-c88b3003e065-tmp-dir\") pod \"node-resolver-kblfw\" (UID: \"62b003b3-9dd3-4f30-b0ac-c88b3003e065\") " pod="openshift-dns/node-resolver-kblfw"
Aug 04 20:40:55.202348 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-etc-openvswitch\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.202698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b89dd0c-99b7-4dce-96c6-3f8e90482915-cni-binary-copy\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.202698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-sys\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.202698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-run-netns\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.202698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202471    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-log-socket\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.202698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202472    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f558bf91-5880-4917-9e1e-c78ae5493458-serviceca\") pod \"node-ca-w7fs7\" (UID: \"f558bf91-5880-4917-9e1e-c78ae5493458\") " pod="openshift-image-registry/node-ca-w7fs7"
Aug 04 20:40:55.202698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202492    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-run-ovn-kubernetes\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.202698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7876b090-ff49-42b7-aff9-93b9aa25ef12-konnectivity-ca\") pod \"konnectivity-agent-vh5t2\" (UID: \"7876b090-ff49-42b7-aff9-93b9aa25ef12\") " pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:40:55.202698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9b89dd0c-99b7-4dce-96c6-3f8e90482915-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.202698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202601    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.202698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-sys-fs\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.202698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-cni-dir\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-run-openvswitch\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-registration-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-kubernetes\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202813    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzzb6\" (UniqueName: \"kubernetes.io/projected/62b003b3-9dd3-4f30-b0ac-c88b3003e065-kube-api-access-wzzb6\") pod \"node-resolver-kblfw\" (UID: \"62b003b3-9dd3-4f30-b0ac-c88b3003e065\") " pod="openshift-dns/node-resolver-kblfw"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-systemd\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-var-lib-openvswitch\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202886    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-device-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f558bf91-5880-4917-9e1e-c78ae5493458-host\") pod \"node-ca-w7fs7\" (UID: \"f558bf91-5880-4917-9e1e-c78ae5493458\") " pod="openshift-image-registry/node-ca-w7fs7"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.202989    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-sysctl-d\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.203032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-lib-modules\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.203045    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f558bf91-5880-4917-9e1e-c78ae5493458-host\") pod \"node-ca-w7fs7\" (UID: \"f558bf91-5880-4917-9e1e-c78ae5493458\") " pod="openshift-image-registry/node-ca-w7fs7"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.203071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-host\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.203094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtw4h\" (UniqueName: \"kubernetes.io/projected/a42c0953-c4c7-4aed-b344-9b63c27267bb-kube-api-access-jtw4h\") pod \"iptables-alerter-w5ftp\" (UID: \"a42c0953-c4c7-4aed-b344-9b63c27267bb\") " pod="openshift-network-operator/iptables-alerter-w5ftp"
Aug 04 20:40:55.203134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.203131    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62b003b3-9dd3-4f30-b0ac-c88b3003e065-hosts-file\") pod \"node-resolver-kblfw\" (UID: \"62b003b3-9dd3-4f30-b0ac-c88b3003e065\") " pod="openshift-dns/node-resolver-kblfw"
Aug 04 20:40:55.203633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.203156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-modprobe-d\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.203633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.203173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62b003b3-9dd3-4f30-b0ac-c88b3003e065-hosts-file\") pod \"node-resolver-kblfw\" (UID: \"62b003b3-9dd3-4f30-b0ac-c88b3003e065\") " pod="openshift-dns/node-resolver-kblfw"
Aug 04 20:40:55.203633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.203178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-socket-dir-parent\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.203633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.203209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-run-systemd\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.203633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.203233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.203633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.203256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9b89dd0c-99b7-4dce-96c6-3f8e90482915-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.209766 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.209748    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 04 20:40:55.213275 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.213253    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzzb6\" (UniqueName: \"kubernetes.io/projected/62b003b3-9dd3-4f30-b0ac-c88b3003e065-kube-api-access-wzzb6\") pod \"node-resolver-kblfw\" (UID: \"62b003b3-9dd3-4f30-b0ac-c88b3003e065\") " pod="openshift-dns/node-resolver-kblfw"
Aug 04 20:40:55.213430 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.213394    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwfc2\" (UniqueName: \"kubernetes.io/projected/d0178830-4009-4007-9829-a2c0aee48925-kube-api-access-bwfc2\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:40:55.213561 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.213539    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npcnj\" (UniqueName: \"kubernetes.io/projected/f558bf91-5880-4917-9e1e-c78ae5493458-kube-api-access-npcnj\") pod \"node-ca-w7fs7\" (UID: \"f558bf91-5880-4917-9e1e-c78ae5493458\") " pod="openshift-image-registry/node-ca-w7fs7"
Aug 04 20:40:55.243405 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.243376    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-249.ec2.internal" event={"ID":"f3d43927280795fd50a92f8253826fd4","Type":"ContainerStarted","Data":"87802b1dd2fa2aa0cd5fff06e117e183eb0180cc6dd123e0ebcca74cbb2d3688"}
Aug 04 20:40:55.303722 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303689    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6g28\" (UniqueName: \"kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28\") pod \"network-check-target-bkp8v\" (UID: \"c1997346-3963-43df-b2f3-515df3356344\") " pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:40:55.303846 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7876b090-ff49-42b7-aff9-93b9aa25ef12-agent-certs\") pod \"konnectivity-agent-vh5t2\" (UID: \"7876b090-ff49-42b7-aff9-93b9aa25ef12\") " pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:40:55.303846 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a42c0953-c4c7-4aed-b344-9b63c27267bb-host-slash\") pod \"iptables-alerter-w5ftp\" (UID: \"a42c0953-c4c7-4aed-b344-9b63c27267bb\") " pod="openshift-network-operator/iptables-alerter-w5ftp"
Aug 04 20:40:55.303846 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303779    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2eea297c-3921-4073-a019-78634486f6dc-etc-tuned\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.303846 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-var-lib-cni-bin\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.303846 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-var-lib-kubelet\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.303846 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303845    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-conf-dir\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.304092 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-etc-kubernetes\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.304092 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-sysctl-conf\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.304092 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-hostroot\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.304092 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303929    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-daemon-config\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.304092 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.303968    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-var-lib-kubelet\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.304092 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-node-log\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.304092 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-slash\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.304092 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-run-netns\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.304092 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a951e2c-81e0-4776-9f94-e4693a89d219-ovnkube-script-lib\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjzfd\" (UniqueName: \"kubernetes.io/projected/283ad954-19e6-44b0-8c87-bb257b1e4a97-kube-api-access-zjzfd\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304153    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2eea297c-3921-4073-a019-78634486f6dc-tmp\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5t24\" (UniqueName: \"kubernetes.io/projected/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-kube-api-access-k5t24\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a951e2c-81e0-4776-9f94-e4693a89d219-ovnkube-config\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304224    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a42c0953-c4c7-4aed-b344-9b63c27267bb-iptables-alerter-script\") pod \"iptables-alerter-w5ftp\" (UID: \"a42c0953-c4c7-4aed-b344-9b63c27267bb\") " pod="openshift-network-operator/iptables-alerter-w5ftp"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrsbg\" (UniqueName: \"kubernetes.io/projected/9b89dd0c-99b7-4dce-96c6-3f8e90482915-kube-api-access-wrsbg\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-var-lib-kubelet\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-etc-openvswitch\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b89dd0c-99b7-4dce-96c6-3f8e90482915-cni-binary-copy\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-sys\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-run-netns\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-log-socket\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-run-ovn-kubernetes\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7876b090-ff49-42b7-aff9-93b9aa25ef12-konnectivity-ca\") pod \"konnectivity-agent-vh5t2\" (UID: \"7876b090-ff49-42b7-aff9-93b9aa25ef12\") " pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304423    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-daemon-config\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.304436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9b89dd0c-99b7-4dce-96c6-3f8e90482915-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.305014 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304488    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-conf-dir\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.305014 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304530    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-etc-kubernetes\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.305014 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304631    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-sysctl-conf\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.305014 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304660    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-hostroot\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.305014 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304906    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9b89dd0c-99b7-4dce-96c6-3f8e90482915-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.305014 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.305014 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-sys-fs\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.305014 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.304985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-cni-dir\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.305014 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-run-openvswitch\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-registration-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-kubernetes\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-systemd\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-var-lib-openvswitch\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-device-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-sysctl-d\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-lib-modules\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-host\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305209    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtw4h\" (UniqueName: \"kubernetes.io/projected/a42c0953-c4c7-4aed-b344-9b63c27267bb-kube-api-access-jtw4h\") pod \"iptables-alerter-w5ftp\" (UID: \"a42c0953-c4c7-4aed-b344-9b63c27267bb\") " pod="openshift-network-operator/iptables-alerter-w5ftp"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-modprobe-d\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-socket-dir-parent\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-run-systemd\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9b89dd0c-99b7-4dce-96c6-3f8e90482915-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-socket-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-os-release\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.305422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-run-multus-certs\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-run-ovn\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305442    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a951e2c-81e0-4776-9f94-e4693a89d219-env-overrides\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-dbus\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-system-cni-dir\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-cnibin\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-system-cni-dir\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-cni-binary-copy\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-var-lib-cni-multus\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-kubelet\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhstm\" (UniqueName: \"kubernetes.io/projected/4a951e2c-81e0-4776-9f94-e4693a89d219-kube-api-access-zhstm\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-etc-selinux\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-kubelet-config\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-run\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8jwg\" (UniqueName: \"kubernetes.io/projected/2eea297c-3921-4073-a019-78634486f6dc-kube-api-access-l8jwg\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305774    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-cnibin\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.306023 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-run-k8s-cni-cncf-io\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-systemd-units\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305845    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-cni-netd\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a951e2c-81e0-4776-9f94-e4693a89d219-ovn-node-metrics-cert\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305923    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-os-release\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-sysconfig\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305978    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-cni-bin\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.306072    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-cni-bin\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.306235    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-var-lib-cni-bin\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.306284    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-node-log\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.306337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-slash\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.306373    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-run-netns\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.306964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.306546    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a951e2c-81e0-4776-9f94-e4693a89d219-ovnkube-config\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.307549 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.306993    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a42c0953-c4c7-4aed-b344-9b63c27267bb-iptables-alerter-script\") pod \"iptables-alerter-w5ftp\" (UID: \"a42c0953-c4c7-4aed-b344-9b63c27267bb\") " pod="openshift-network-operator/iptables-alerter-w5ftp"
Aug 04 20:40:55.307549 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-var-lib-kubelet\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.307549 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-os-release\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.307549 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307359    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-socket-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.307549 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307371    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-run-multus-certs\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.307549 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307413    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-run-ovn\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.307549 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307418    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-etc-openvswitch\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.307549 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.307891 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307644    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-dbus\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:55.307891 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307675    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a951e2c-81e0-4776-9f94-e4693a89d219-ovnkube-script-lib\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.307891 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-system-cni-dir\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.307891 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307731    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-cnibin\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.307891 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307764    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-run\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.307891 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307820    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-system-cni-dir\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.307891 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b89dd0c-99b7-4dce-96c6-3f8e90482915-cni-binary-copy\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.308160 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.307964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-cnibin\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.308160 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-run-k8s-cni-cncf-io\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.308160 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.308059    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 20:40:55.308160 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308095    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-cni-netd\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.308160 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.308114    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret podName:6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33 nodeName:}" failed. No retries permitted until 2026-08-04 20:40:55.808095711 +0000 UTC m=+3.051715004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret") pod "global-pull-secret-syncer-62hnz" (UID: "6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33") : object "kube-system"/"original-pull-secret" not registered
Aug 04 20:40:55.308160 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308059    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-systemd-units\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.308452 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a951e2c-81e0-4776-9f94-e4693a89d219-env-overrides\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.308452 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-sys\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.308452 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-cni-binary-copy\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.308452 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.305555    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a42c0953-c4c7-4aed-b344-9b63c27267bb-host-slash\") pod \"iptables-alerter-w5ftp\" (UID: \"a42c0953-c4c7-4aed-b344-9b63c27267bb\") " pod="openshift-network-operator/iptables-alerter-w5ftp"
Aug 04 20:40:55.308452 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-run-netns\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.308452 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-log-socket\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.308452 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-run-ovn-kubernetes\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.308791 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-os-release\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.308791 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308555    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-run-openvswitch\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.308791 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308560    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-sysctl-d\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.308791 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-host\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.308791 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308731    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-host-var-lib-cni-multus\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.308791 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-host-kubelet\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.309074 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308885    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-modprobe-d\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.309074 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-run-systemd\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.309074 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.308959    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-device-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.309074 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-etc-selinux\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.309074 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309009    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7876b090-ff49-42b7-aff9-93b9aa25ef12-konnectivity-ca\") pod \"konnectivity-agent-vh5t2\" (UID: \"7876b090-ff49-42b7-aff9-93b9aa25ef12\") " pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:40:55.309074 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309071    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b89dd0c-99b7-4dce-96c6-3f8e90482915-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.309349 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-systemd\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.309349 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309074    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-registration-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.309349 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309095    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-lib-modules\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.309349 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-kubernetes\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.309349 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309022    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-socket-dir-parent\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.309349 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a951e2c-81e0-4776-9f94-e4693a89d219-var-lib-openvswitch\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.309349 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-kubelet-config\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:55.309349 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309182    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-multus-cni-dir\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.309349 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309214    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-sys-fs\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.309349 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309221    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/283ad954-19e6-44b0-8c87-bb257b1e4a97-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.309349 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2eea297c-3921-4073-a019-78634486f6dc-etc-sysconfig\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.309895 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.309541    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9b89dd0c-99b7-4dce-96c6-3f8e90482915-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.310559 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.310129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2eea297c-3921-4073-a019-78634486f6dc-etc-tuned\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.310559 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.310472    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7876b090-ff49-42b7-aff9-93b9aa25ef12-agent-certs\") pod \"konnectivity-agent-vh5t2\" (UID: \"7876b090-ff49-42b7-aff9-93b9aa25ef12\") " pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:40:55.310705 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.310651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2eea297c-3921-4073-a019-78634486f6dc-tmp\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.310800 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.310785    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a951e2c-81e0-4776-9f94-e4693a89d219-ovn-node-metrics-cert\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.327932 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.327872    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 20:40:55.327932 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.327901    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 20:40:55.327932 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.327915    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f6g28 for pod openshift-network-diagnostics/network-check-target-bkp8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:40:55.328106 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.328000    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28 podName:c1997346-3963-43df-b2f3-515df3356344 nodeName:}" failed. No retries permitted until 2026-08-04 20:40:55.827982621 +0000 UTC m=+3.071601909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f6g28" (UniqueName: "kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28") pod "network-check-target-bkp8v" (UID: "c1997346-3963-43df-b2f3-515df3356344") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:40:55.329463 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.329418    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5t24\" (UniqueName: \"kubernetes.io/projected/568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc-kube-api-access-k5t24\") pod \"multus-mk5vg\" (UID: \"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc\") " pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.329696 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.329652    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8jwg\" (UniqueName: \"kubernetes.io/projected/2eea297c-3921-4073-a019-78634486f6dc-kube-api-access-l8jwg\") pod \"tuned-89np2\" (UID: \"2eea297c-3921-4073-a019-78634486f6dc\") " pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.330680 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.330656    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtw4h\" (UniqueName: \"kubernetes.io/projected/a42c0953-c4c7-4aed-b344-9b63c27267bb-kube-api-access-jtw4h\") pod \"iptables-alerter-w5ftp\" (UID: \"a42c0953-c4c7-4aed-b344-9b63c27267bb\") " pod="openshift-network-operator/iptables-alerter-w5ftp"
Aug 04 20:40:55.330680 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.330672    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjzfd\" (UniqueName: \"kubernetes.io/projected/283ad954-19e6-44b0-8c87-bb257b1e4a97-kube-api-access-zjzfd\") pod \"aws-ebs-csi-driver-node-f2lk7\" (UID: \"283ad954-19e6-44b0-8c87-bb257b1e4a97\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.330856 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.330837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrsbg\" (UniqueName: \"kubernetes.io/projected/9b89dd0c-99b7-4dce-96c6-3f8e90482915-kube-api-access-wrsbg\") pod \"multus-additional-cni-plugins-rpwv5\" (UID: \"9b89dd0c-99b7-4dce-96c6-3f8e90482915\") " pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.330953 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.330935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhstm\" (UniqueName: \"kubernetes.io/projected/4a951e2c-81e0-4776-9f94-e4693a89d219-kube-api-access-zhstm\") pod \"ovnkube-node-8rgcs\" (UID: \"4a951e2c-81e0-4776-9f94-e4693a89d219\") " pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.401211 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.401184    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w7fs7"
Aug 04 20:40:55.407843 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.407816    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kblfw"
Aug 04 20:40:55.416390 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.416371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:40:55.422901 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.422883    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-89np2"
Aug 04 20:40:55.436375 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.436353    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w5ftp"
Aug 04 20:40:55.444842 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.444828    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rpwv5"
Aug 04 20:40:55.452471 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.452452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mk5vg"
Aug 04 20:40:55.459898 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.459881    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:40:55.466410 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.466393    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7"
Aug 04 20:40:55.531415 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.531387    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 20:40:55.709310 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.709236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:40:55.709459 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.709389    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:40:55.709459 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.709455    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs podName:d0178830-4009-4007-9829-a2c0aee48925 nodeName:}" failed. No retries permitted until 2026-08-04 20:40:56.709433292 +0000 UTC m=+3.953052553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs") pod "network-metrics-daemon-8wdv9" (UID: "d0178830-4009-4007-9829-a2c0aee48925") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:40:55.809634 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.809606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:55.809789 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.809741    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 20:40:55.809830 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.809805    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret podName:6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33 nodeName:}" failed. No retries permitted until 2026-08-04 20:40:56.809791704 +0000 UTC m=+4.053410964 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret") pod "global-pull-secret-syncer-62hnz" (UID: "6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33") : object "kube-system"/"original-pull-secret" not registered
Aug 04 20:40:55.910963 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:55.910928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6g28\" (UniqueName: \"kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28\") pod \"network-check-target-bkp8v\" (UID: \"c1997346-3963-43df-b2f3-515df3356344\") " pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:40:55.911139 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.911119    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 20:40:55.911187 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.911145    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 20:40:55.911187 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.911161    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f6g28 for pod openshift-network-diagnostics/network-check-target-bkp8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:40:55.911269 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:55.911230    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28 podName:c1997346-3963-43df-b2f3-515df3356344 nodeName:}" failed. No retries permitted until 2026-08-04 20:40:56.911211448 +0000 UTC m=+4.154830725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f6g28" (UniqueName: "kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28") pod "network-check-target-bkp8v" (UID: "c1997346-3963-43df-b2f3-515df3356344") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:40:56.134824 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:56.134802    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eea297c_3921_4073_a019_78634486f6dc.slice/crio-9ad04c3de95d0b2d7b66ca01b16e002bcb97e03e7c411164ce837e5288f9066a WatchSource:0}: Error finding container 9ad04c3de95d0b2d7b66ca01b16e002bcb97e03e7c411164ce837e5288f9066a: Status 404 returned error can't find the container with id 9ad04c3de95d0b2d7b66ca01b16e002bcb97e03e7c411164ce837e5288f9066a
Aug 04 20:40:56.137289 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:56.137260    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a951e2c_81e0_4776_9f94_e4693a89d219.slice/crio-112d5a439829e43184527d91eafffb16453d11577c992d1c3e6e48b0a3084035 WatchSource:0}: Error finding container 112d5a439829e43184527d91eafffb16453d11577c992d1c3e6e48b0a3084035: Status 404 returned error can't find the container with id 112d5a439829e43184527d91eafffb16453d11577c992d1c3e6e48b0a3084035
Aug 04 20:40:56.138725 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:56.138700    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568fdcaf_8d26_44c9_89fa_9ab8ce5e8bbc.slice/crio-f9beb316faea4d5b76320a873e279226fb3fb75c8e5094735736569f5abe66a3 WatchSource:0}: Error finding container f9beb316faea4d5b76320a873e279226fb3fb75c8e5094735736569f5abe66a3: Status 404 returned error can't find the container with id f9beb316faea4d5b76320a873e279226fb3fb75c8e5094735736569f5abe66a3
Aug 04 20:40:56.139982 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:56.139960    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42c0953_c4c7_4aed_b344_9b63c27267bb.slice/crio-6249647bb464cfa81c320054f934dda608571f0f04e4d93597af26bf7530e026 WatchSource:0}: Error finding container 6249647bb464cfa81c320054f934dda608571f0f04e4d93597af26bf7530e026: Status 404 returned error can't find the container with id 6249647bb464cfa81c320054f934dda608571f0f04e4d93597af26bf7530e026
Aug 04 20:40:56.142043 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:56.141651    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b89dd0c_99b7_4dce_96c6_3f8e90482915.slice/crio-fae1c3f31df62935318f314b486079f97b1b561f1912dac958cfdcaa57ad463c WatchSource:0}: Error finding container fae1c3f31df62935318f314b486079f97b1b561f1912dac958cfdcaa57ad463c: Status 404 returned error can't find the container with id fae1c3f31df62935318f314b486079f97b1b561f1912dac958cfdcaa57ad463c
Aug 04 20:40:56.142532 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:56.142510    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf558bf91_5880_4917_9e1e_c78ae5493458.slice/crio-241f9018b7054e418529d85292bf2630a8a368cbcdd8b0370dbbc993618603af WatchSource:0}: Error finding container 241f9018b7054e418529d85292bf2630a8a368cbcdd8b0370dbbc993618603af: Status 404 returned error can't find the container with id 241f9018b7054e418529d85292bf2630a8a368cbcdd8b0370dbbc993618603af
Aug 04 20:40:56.143367 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:56.143283    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b003b3_9dd3_4f30_b0ac_c88b3003e065.slice/crio-072779a12bc4c2414a2234d754189167015b5003c64582309da15038a7a86ad3 WatchSource:0}: Error finding container 072779a12bc4c2414a2234d754189167015b5003c64582309da15038a7a86ad3: Status 404 returned error can't find the container with id 072779a12bc4c2414a2234d754189167015b5003c64582309da15038a7a86ad3
Aug 04 20:40:56.144241 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:56.143998    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod283ad954_19e6_44b0_8c87_bb257b1e4a97.slice/crio-f8afd102eba96d0b56a1f9bafa5b9eb21bb70f5e6cb93d0033cd07cd85a0a979 WatchSource:0}: Error finding container f8afd102eba96d0b56a1f9bafa5b9eb21bb70f5e6cb93d0033cd07cd85a0a979: Status 404 returned error can't find the container with id f8afd102eba96d0b56a1f9bafa5b9eb21bb70f5e6cb93d0033cd07cd85a0a979
Aug 04 20:40:56.146842 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:40:56.146810    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7876b090_ff49_42b7_aff9_93b9aa25ef12.slice/crio-62e5b3bfae467d98a1b069c987e29bd3b82b0bb1630c7f61d1ff5cc81496b42d WatchSource:0}: Error finding container 62e5b3bfae467d98a1b069c987e29bd3b82b0bb1630c7f61d1ff5cc81496b42d: Status 404 returned error can't find the container with id 62e5b3bfae467d98a1b069c987e29bd3b82b0bb1630c7f61d1ff5cc81496b42d
Aug 04 20:40:56.161583 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.161513    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 20:35:55 +0000 UTC" deadline="2028-03-20 02:06:06.967561611 +0000 UTC"
Aug 04 20:40:56.161583 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.161539    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14237h25m10.806026485s"
Aug 04 20:40:56.237669 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.237546    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:40:56.237787 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:56.237764    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:40:56.246065 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.246041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs" event={"ID":"4a951e2c-81e0-4776-9f94-e4693a89d219","Type":"ContainerStarted","Data":"112d5a439829e43184527d91eafffb16453d11577c992d1c3e6e48b0a3084035"}
Aug 04 20:40:56.247050 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.247030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-89np2" event={"ID":"2eea297c-3921-4073-a019-78634486f6dc","Type":"ContainerStarted","Data":"9ad04c3de95d0b2d7b66ca01b16e002bcb97e03e7c411164ce837e5288f9066a"}
Aug 04 20:40:56.248109 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.248086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vh5t2" event={"ID":"7876b090-ff49-42b7-aff9-93b9aa25ef12","Type":"ContainerStarted","Data":"62e5b3bfae467d98a1b069c987e29bd3b82b0bb1630c7f61d1ff5cc81496b42d"}
Aug 04 20:40:56.249118 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.249096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7" event={"ID":"283ad954-19e6-44b0-8c87-bb257b1e4a97","Type":"ContainerStarted","Data":"f8afd102eba96d0b56a1f9bafa5b9eb21bb70f5e6cb93d0033cd07cd85a0a979"}
Aug 04 20:40:56.250123 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.250103    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w7fs7" event={"ID":"f558bf91-5880-4917-9e1e-c78ae5493458","Type":"ContainerStarted","Data":"241f9018b7054e418529d85292bf2630a8a368cbcdd8b0370dbbc993618603af"}
Aug 04 20:40:56.251116 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.251097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rpwv5" event={"ID":"9b89dd0c-99b7-4dce-96c6-3f8e90482915","Type":"ContainerStarted","Data":"fae1c3f31df62935318f314b486079f97b1b561f1912dac958cfdcaa57ad463c"}
Aug 04 20:40:56.252305 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.252287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mk5vg" event={"ID":"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc","Type":"ContainerStarted","Data":"f9beb316faea4d5b76320a873e279226fb3fb75c8e5094735736569f5abe66a3"}
Aug 04 20:40:56.254188 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.254168    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-249.ec2.internal" event={"ID":"f3d43927280795fd50a92f8253826fd4","Type":"ContainerStarted","Data":"e0040a25c7d993fda618f36f598b0f210241a58a4107a2f78bc4fc5a6c354df2"}
Aug 04 20:40:56.255248 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.255229    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kblfw" event={"ID":"62b003b3-9dd3-4f30-b0ac-c88b3003e065","Type":"ContainerStarted","Data":"072779a12bc4c2414a2234d754189167015b5003c64582309da15038a7a86ad3"}
Aug 04 20:40:56.256197 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.256180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w5ftp" event={"ID":"a42c0953-c4c7-4aed-b344-9b63c27267bb","Type":"ContainerStarted","Data":"6249647bb464cfa81c320054f934dda608571f0f04e4d93597af26bf7530e026"}
Aug 04 20:40:56.271825 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.271780    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-249.ec2.internal" podStartSLOduration=2.271768654 podStartE2EDuration="2.271768654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:40:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:40:56.271552471 +0000 UTC m=+3.515171752" watchObservedRunningTime="2026-08-04 20:40:56.271768654 +0000 UTC m=+3.515387935"
Aug 04 20:40:56.715229 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.715138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:40:56.715435 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:56.715375    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:40:56.715504 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:56.715440    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs podName:d0178830-4009-4007-9829-a2c0aee48925 nodeName:}" failed. No retries permitted until 2026-08-04 20:40:58.715418364 +0000 UTC m=+5.959037629 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs") pod "network-metrics-daemon-8wdv9" (UID: "d0178830-4009-4007-9829-a2c0aee48925") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:40:56.815801 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.815770    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:56.815965 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:56.815876    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 20:40:56.815965 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:56.815928    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret podName:6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33 nodeName:}" failed. No retries permitted until 2026-08-04 20:40:58.81591148 +0000 UTC m=+6.059530759 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret") pod "global-pull-secret-syncer-62hnz" (UID: "6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33") : object "kube-system"/"original-pull-secret" not registered
Aug 04 20:40:56.916907 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:56.916756    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6g28\" (UniqueName: \"kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28\") pod \"network-check-target-bkp8v\" (UID: \"c1997346-3963-43df-b2f3-515df3356344\") " pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:40:56.917035 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:56.916938    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 20:40:56.917035 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:56.916956    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 20:40:56.917035 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:56.916969    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f6g28 for pod openshift-network-diagnostics/network-check-target-bkp8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:40:56.917035 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:56.917026    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28 podName:c1997346-3963-43df-b2f3-515df3356344 nodeName:}" failed. No retries permitted until 2026-08-04 20:40:58.917007743 +0000 UTC m=+6.160627017 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f6g28" (UniqueName: "kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28") pod "network-check-target-bkp8v" (UID: "c1997346-3963-43df-b2f3-515df3356344") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:40:57.162037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:57.161953    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 20:35:55 +0000 UTC" deadline="2028-02-07 00:06:17.761292046 +0000 UTC"
Aug 04 20:40:57.162037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:57.162012    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13227h25m20.599283808s"
Aug 04 20:40:57.242202 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:57.241589    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:40:57.242202 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:57.241715    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:40:57.242202 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:57.241784    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:57.242202 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:57.241853    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:40:57.273207 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:57.273178    2563 generic.go:356] "Generic (PLEG): container finished" podID="3d8aaa7b15afe68782f913fe00a30060" containerID="79bb69661faee047eb6d9c158525dda87bd3f0d614bd65161c539bf706b02776" exitCode=0
Aug 04 20:40:57.273863 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:57.273602    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal" event={"ID":"3d8aaa7b15afe68782f913fe00a30060","Type":"ContainerDied","Data":"79bb69661faee047eb6d9c158525dda87bd3f0d614bd65161c539bf706b02776"}
Aug 04 20:40:58.236827 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:58.236800    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:40:58.237273 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:58.236919    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:40:58.299485 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:58.299448    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal" event={"ID":"3d8aaa7b15afe68782f913fe00a30060","Type":"ContainerStarted","Data":"db1625608c17d1202da6698cc7f34272d7a922d62c827c88f22a81c1c5caaaac"}
Aug 04 20:40:58.738830 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:58.738799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:40:58.739006 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:58.738940    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:40:58.739006 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:58.738999    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs podName:d0178830-4009-4007-9829-a2c0aee48925 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:02.738982021 +0000 UTC m=+9.982601295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs") pod "network-metrics-daemon-8wdv9" (UID: "d0178830-4009-4007-9829-a2c0aee48925") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:40:58.839765 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:58.839729    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:58.839934 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:58.839901    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 20:40:58.839987 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:58.839963    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret podName:6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:02.839943959 +0000 UTC m=+10.083563272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret") pod "global-pull-secret-syncer-62hnz" (UID: "6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33") : object "kube-system"/"original-pull-secret" not registered
Aug 04 20:40:58.940534 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:58.940498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6g28\" (UniqueName: \"kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28\") pod \"network-check-target-bkp8v\" (UID: \"c1997346-3963-43df-b2f3-515df3356344\") " pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:40:58.940719 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:58.940700    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 20:40:58.940803 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:58.940729    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 20:40:58.940803 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:58.940742    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f6g28 for pod openshift-network-diagnostics/network-check-target-bkp8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:40:58.940803 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:58.940798    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28 podName:c1997346-3963-43df-b2f3-515df3356344 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:02.940781816 +0000 UTC m=+10.184401080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f6g28" (UniqueName: "kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28") pod "network-check-target-bkp8v" (UID: "c1997346-3963-43df-b2f3-515df3356344") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:40:59.238507 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:59.238417    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:40:59.238913 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:59.238553    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:40:59.240471 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:40:59.240448    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:40:59.240605 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:40:59.240567    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:00.237157 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:00.237122    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:00.237359 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:00.237244    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:01.237329 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:01.237275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:01.237762 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:01.237413    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:01.237831 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:01.237289    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:01.237890 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:01.237870    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:02.237697 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:02.237665    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:02.238109 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:02.237770    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:02.770380 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:02.769824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:02.770380 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:02.769984    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:41:02.770380 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:02.770044    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs podName:d0178830-4009-4007-9829-a2c0aee48925 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:10.770026098 +0000 UTC m=+18.013645363 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs") pod "network-metrics-daemon-8wdv9" (UID: "d0178830-4009-4007-9829-a2c0aee48925") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:41:02.870553 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:02.870523    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:02.870716 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:02.870681    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 20:41:02.870791 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:02.870738    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret podName:6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:10.870721098 +0000 UTC m=+18.114340371 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret") pod "global-pull-secret-syncer-62hnz" (UID: "6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33") : object "kube-system"/"original-pull-secret" not registered
Aug 04 20:41:02.971580 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:02.971544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6g28\" (UniqueName: \"kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28\") pod \"network-check-target-bkp8v\" (UID: \"c1997346-3963-43df-b2f3-515df3356344\") " pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:02.971754 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:02.971733    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 20:41:02.971754 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:02.971753    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 20:41:02.971851 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:02.971766    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f6g28 for pod openshift-network-diagnostics/network-check-target-bkp8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:41:02.971851 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:02.971842    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28 podName:c1997346-3963-43df-b2f3-515df3356344 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:10.971806189 +0000 UTC m=+18.215425464 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f6g28" (UniqueName: "kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28") pod "network-check-target-bkp8v" (UID: "c1997346-3963-43df-b2f3-515df3356344") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:41:03.238287 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:03.238205    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:03.238682 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:03.238346    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:03.238682 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:03.238385    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:03.238682 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:03.238493    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:04.237791 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:04.237259    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:04.237791 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:04.237410    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:05.240438 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:05.239740    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:05.240438 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:05.239875    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:05.240438 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:05.240262    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:05.240438 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:05.240385    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:06.237263 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:06.237231    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:06.237425 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:06.237360    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:07.237202 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:07.237172    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:07.237670 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:07.237303    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:07.237670 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:07.237476    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:07.237670 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:07.237558    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:08.236997 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:08.236971    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:08.237141 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:08.237090    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:09.237394 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:09.236871    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:09.237394 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:09.237022    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:09.237394 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:09.237079    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:09.237394 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:09.237196    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:10.237529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:10.237496    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:10.238042 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:10.237610    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:10.835989 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:10.835959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:10.836147 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:10.836103    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:41:10.836196 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:10.836177    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs podName:d0178830-4009-4007-9829-a2c0aee48925 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:26.836160153 +0000 UTC m=+34.079779419 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs") pod "network-metrics-daemon-8wdv9" (UID: "d0178830-4009-4007-9829-a2c0aee48925") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:41:10.937000 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:10.936967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:10.937163 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:10.937077    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 20:41:10.937163 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:10.937148    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret podName:6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:26.937127257 +0000 UTC m=+34.180746519 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret") pod "global-pull-secret-syncer-62hnz" (UID: "6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33") : object "kube-system"/"original-pull-secret" not registered
Aug 04 20:41:11.037782 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:11.037743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6g28\" (UniqueName: \"kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28\") pod \"network-check-target-bkp8v\" (UID: \"c1997346-3963-43df-b2f3-515df3356344\") " pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:11.037951 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:11.037931    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 20:41:11.037993 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:11.037958    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 20:41:11.037993 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:11.037967    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f6g28 for pod openshift-network-diagnostics/network-check-target-bkp8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:41:11.038067 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:11.038012    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28 podName:c1997346-3963-43df-b2f3-515df3356344 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:27.03799799 +0000 UTC m=+34.281617250 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f6g28" (UniqueName: "kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28") pod "network-check-target-bkp8v" (UID: "c1997346-3963-43df-b2f3-515df3356344") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:41:11.237243 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:11.237165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:11.237412 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:11.237171    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:11.237412 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:11.237293    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:11.237412 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:11.237372    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:12.237366 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:12.237333    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:12.237862 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:12.237450    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:13.237982 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:13.237937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:13.238528 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:13.238064    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:13.238528 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:13.238149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:13.238528 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:13.238252    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:14.236948 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:14.236921    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:14.237099 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:14.237037    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:15.237305 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.236902    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:15.238060 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:15.237438    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:15.238060 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.236976    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:15.238060 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:15.237643    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:15.338642 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.338614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kblfw" event={"ID":"62b003b3-9dd3-4f30-b0ac-c88b3003e065","Type":"ContainerStarted","Data":"5d75c1c43a7a517ca1d4c4d6d7ba8aa3c95b9b3c94414b1acc2a5ec30b03141a"}
Aug 04 20:41:15.343400 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.341936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs" event={"ID":"4a951e2c-81e0-4776-9f94-e4693a89d219","Type":"ContainerStarted","Data":"600315af0135158782853ab75d66d8549aeea8eaf41a5ed394e697370ae069a4"}
Aug 04 20:41:15.343400 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.342117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs" event={"ID":"4a951e2c-81e0-4776-9f94-e4693a89d219","Type":"ContainerStarted","Data":"116348730f00fa1a22df2e3306a762b523aaa8058cea3a2b0ebd8da4f710c30a"}
Aug 04 20:41:15.343400 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.342214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs" event={"ID":"4a951e2c-81e0-4776-9f94-e4693a89d219","Type":"ContainerStarted","Data":"03fc7f1ec06f29e7e24d0ec285390713a4eebc43d811a882458a92e532bf1391"}
Aug 04 20:41:15.343400 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.342268    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs" event={"ID":"4a951e2c-81e0-4776-9f94-e4693a89d219","Type":"ContainerStarted","Data":"50a105e5c499c570728573508d100e8b7b534d1363eb584603460e654bcc43f8"}
Aug 04 20:41:15.343400 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.342367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs" event={"ID":"4a951e2c-81e0-4776-9f94-e4693a89d219","Type":"ContainerStarted","Data":"8d4b228a295536b5556fe68a2afef5457539e76f2ed1e430c05174a4e8d456ad"}
Aug 04 20:41:15.344692 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.344663    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-89np2" event={"ID":"2eea297c-3921-4073-a019-78634486f6dc","Type":"ContainerStarted","Data":"672805ac1569c3fc418768263a6d19aa5f9ad8d0ca02d033c2e51884b5f84fb5"}
Aug 04 20:41:15.345941 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.345921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vh5t2" event={"ID":"7876b090-ff49-42b7-aff9-93b9aa25ef12","Type":"ContainerStarted","Data":"9ec8e6f8ff1f30b6c039d205630399ed27ca5a5f85ffbc72bf4658d20d22036b"}
Aug 04 20:41:15.347342 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.347306    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7" event={"ID":"283ad954-19e6-44b0-8c87-bb257b1e4a97","Type":"ContainerStarted","Data":"ed69b5244c22f4a9e10d04cdc07b9cf9928987fbe6d539ff11324b7a308b7ce7"}
Aug 04 20:41:15.348611 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.348580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w7fs7" event={"ID":"f558bf91-5880-4917-9e1e-c78ae5493458","Type":"ContainerStarted","Data":"66d8e548d1c6e09fb2f0ad415e5e7b596119792cf030153dd3c3d5cb90cad641"}
Aug 04 20:41:15.349934 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.349913    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b89dd0c-99b7-4dce-96c6-3f8e90482915" containerID="d7f981a0cf9447c4ad04ddaaad62956f5ceba2bdaae808f70af44ee3ed8a993c" exitCode=0
Aug 04 20:41:15.350017 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.349992    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rpwv5" event={"ID":"9b89dd0c-99b7-4dce-96c6-3f8e90482915","Type":"ContainerDied","Data":"d7f981a0cf9447c4ad04ddaaad62956f5ceba2bdaae808f70af44ee3ed8a993c"}
Aug 04 20:41:15.351348 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.351274    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mk5vg" event={"ID":"568fdcaf-8d26-44c9-89fa-9ab8ce5e8bbc","Type":"ContainerStarted","Data":"41f25625c09ef3b23f6ce2a970358b68a37e94e8b19786a2f047337ec9dde0cc"}
Aug 04 20:41:15.355910 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.355872    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kblfw" podStartSLOduration=3.807020855 podStartE2EDuration="22.355858223s" totalImagesPullingTime="18.548837368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:15.3555758 +0000 UTC m=+22.599195083" watchObservedRunningTime="2026-08-04 20:41:15.355858223 +0000 UTC m=+22.599477505"
Aug 04 20:41:15.355976 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.355959    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-249.ec2.internal" podStartSLOduration=21.355952795 podStartE2EDuration="21.355952795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:40:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:40:58.317237382 +0000 UTC m=+5.560856663" watchObservedRunningTime="2026-08-04 20:41:15.355952795 +0000 UTC m=+22.599572077"
Aug 04 20:41:15.400354 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.400293    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mk5vg" podStartSLOduration=3.578012819 podStartE2EDuration="22.400282079s" totalImagesPullingTime="18.82226926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:15.400063758 +0000 UTC m=+22.643683040" watchObservedRunningTime="2026-08-04 20:41:15.400282079 +0000 UTC m=+22.643901360"
Aug 04 20:41:15.415881 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.415839    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w7fs7" podStartSLOduration=3.835874565 podStartE2EDuration="22.415822742s" totalImagesPullingTime="18.579948177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:15.415176075 +0000 UTC m=+22.658795355" watchObservedRunningTime="2026-08-04 20:41:15.415822742 +0000 UTC m=+22.659442028"
Aug 04 20:41:15.431172 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.431138    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vh5t2" podStartSLOduration=11.918679995 podStartE2EDuration="22.431111255s" totalImagesPullingTime="10.51243126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:15.430884282 +0000 UTC m=+22.674503575" watchObservedRunningTime="2026-08-04 20:41:15.431111255 +0000 UTC m=+22.674730537"
Aug 04 20:41:15.454938 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:15.454901    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-89np2" podStartSLOduration=3.864814742 podStartE2EDuration="22.454890339s" totalImagesPullingTime="18.590075597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:15.454579177 +0000 UTC m=+22.698198458" watchObservedRunningTime="2026-08-04 20:41:15.454890339 +0000 UTC m=+22.698509620"
Aug 04 20:41:16.237382 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:16.237234    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:16.237382 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:16.237363    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:16.238998 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:16.238968    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 04 20:41:16.355260 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:16.355187    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7" event={"ID":"283ad954-19e6-44b0-8c87-bb257b1e4a97","Type":"ContainerStarted","Data":"53ee6218d492e7ee1f01ab76e82f8fa22f7d681c74753f9b28ef04efa999e496"}
Aug 04 20:41:16.356835 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:16.356801    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w5ftp" event={"ID":"a42c0953-c4c7-4aed-b344-9b63c27267bb","Type":"ContainerStarted","Data":"0778651a2f214f92f04ac551e3ea042287591c3eba799173c587a018d6cc4190"}
Aug 04 20:41:16.360103 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:16.360080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs" event={"ID":"4a951e2c-81e0-4776-9f94-e4693a89d219","Type":"ContainerStarted","Data":"2047c9c09362b1db19f1caa59c1e946bd181553205c436169b011dc49a7e9cd1"}
Aug 04 20:41:16.373156 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:16.373109    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w5ftp" podStartSLOduration=4.7908710580000005 podStartE2EDuration="23.373093693s" totalImagesPullingTime="18.582222635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:16.37204339 +0000 UTC m=+23.615662672" watchObservedRunningTime="2026-08-04 20:41:16.373093693 +0000 UTC m=+23.616712976"
Aug 04 20:41:17.164471 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:17.164373    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T20:41:16.238983699Z","UUID":"558a7348-848a-4f5d-b720-3f9abb09213b","Handler":null,"Name":"","Endpoint":""}
Aug 04 20:41:17.166019 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:17.165973    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 04 20:41:17.166019 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:17.166017    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 04 20:41:17.237269 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:17.237239    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:17.237425 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:17.237379    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:17.237814 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:17.237432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:17.237814 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:17.237515    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:18.237147 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:18.236943    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:18.237358 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:18.237253    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:18.369108 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:18.369071    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs" event={"ID":"4a951e2c-81e0-4776-9f94-e4693a89d219","Type":"ContainerStarted","Data":"ee2fca3791b3d9c50d4bde962654678328c602ec7f8af2553876e0e376990a73"}
Aug 04 20:41:18.371423 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:18.371396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7" event={"ID":"283ad954-19e6-44b0-8c87-bb257b1e4a97","Type":"ContainerStarted","Data":"96d86703fe265d11f1c6483a8359f7f1b807792eba93ec67419a3d5a048443da"}
Aug 04 20:41:18.391211 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:18.391159    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f2lk7" podStartSLOduration=4.524719067 podStartE2EDuration="25.391141983s" totalImagesPullingTime="20.866422916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:18.390745109 +0000 UTC m=+25.634364416" watchObservedRunningTime="2026-08-04 20:41:18.391141983 +0000 UTC m=+25.634761266"
Aug 04 20:41:19.237682 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:19.237643    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:19.237682 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:19.237669    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:19.237880 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:19.237772    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:19.237926 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:19.237893    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:19.939223 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:19.939144    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:41:19.940175 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:19.939826    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:41:20.237470 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:20.237265    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:20.237625 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:20.237540    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:20.379535 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:20.379503    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs" event={"ID":"4a951e2c-81e0-4776-9f94-e4693a89d219","Type":"ContainerStarted","Data":"4f93e01db97cac8556493c3c6719b67a1fed40f11efda40e7608634a95be458d"}
Aug 04 20:41:20.379795 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:20.379767    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:41:20.381424 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:20.381402    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b89dd0c-99b7-4dce-96c6-3f8e90482915" containerID="c49d164ad4ea88bbe5e5d11de770dba4b48937ba84de98c7f38628d3a85b7a96" exitCode=0
Aug 04 20:41:20.381519 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:20.381456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rpwv5" event={"ID":"9b89dd0c-99b7-4dce-96c6-3f8e90482915","Type":"ContainerDied","Data":"c49d164ad4ea88bbe5e5d11de770dba4b48937ba84de98c7f38628d3a85b7a96"}
Aug 04 20:41:20.395640 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:20.395621    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:41:20.420945 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:20.420904    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs" podStartSLOduration=8.797281111 podStartE2EDuration="27.420889605s" totalImagesPullingTime="18.623608494s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:20.420272451 +0000 UTC m=+27.663891733" watchObservedRunningTime="2026-08-04 20:41:20.420889605 +0000 UTC m=+27.664508886"
Aug 04 20:41:20.475433 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:20.475371    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:41:20.475943 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:20.475921    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vh5t2"
Aug 04 20:41:21.237225 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:21.237144    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:21.237833 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:21.237285    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:21.237833 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:21.237669    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:21.237833 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:21.237753    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:21.383991 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:21.383921    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:41:21.384121 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:21.384020    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:41:21.397913 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:21.397889    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:41:22.024174 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:22.023966    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8wdv9"]
Aug 04 20:41:22.024406 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:22.024244    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:22.024488 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:22.024396    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:22.026897 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:22.026872    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bkp8v"]
Aug 04 20:41:22.027054 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:22.026982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:22.027135 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:22.027081    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:22.027517 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:22.027490    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-62hnz"]
Aug 04 20:41:22.027624 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:22.027582    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:22.027708 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:22.027691    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:22.387120 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:22.387047    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b89dd0c-99b7-4dce-96c6-3f8e90482915" containerID="8f922376a7d28aa89de786877318d7f1b9fc3c4015fa61b12576296497cc02bc" exitCode=0
Aug 04 20:41:22.387463 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:22.387122    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rpwv5" event={"ID":"9b89dd0c-99b7-4dce-96c6-3f8e90482915","Type":"ContainerDied","Data":"8f922376a7d28aa89de786877318d7f1b9fc3c4015fa61b12576296497cc02bc"}
Aug 04 20:41:23.237920 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:23.237887    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:23.238074 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:23.237971    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:24.237458 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:24.237421    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:24.237458 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:24.237446    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:24.238136 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:24.237554    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:24.238136 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:24.237669    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:24.392956 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:24.392869    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b89dd0c-99b7-4dce-96c6-3f8e90482915" containerID="89ea23f65feb8af522221ec83d33f3318c206958c9ed0e7bf71e5a36422b9ba8" exitCode=0
Aug 04 20:41:24.392956 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:24.392912    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rpwv5" event={"ID":"9b89dd0c-99b7-4dce-96c6-3f8e90482915","Type":"ContainerDied","Data":"89ea23f65feb8af522221ec83d33f3318c206958c9ed0e7bf71e5a36422b9ba8"}
Aug 04 20:41:25.237092 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:25.237060    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:25.237281 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:25.237192    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:26.237490 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:26.237411    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:26.237856 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:26.237539    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62hnz" podUID="6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33"
Aug 04 20:41:26.237856 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:26.237602    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:26.237856 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:26.237704    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8wdv9" podUID="d0178830-4009-4007-9829-a2c0aee48925"
Aug 04 20:41:26.857377 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:26.857344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:26.857553 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:26.857499    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:41:26.857596 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:26.857561    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs podName:d0178830-4009-4007-9829-a2c0aee48925 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:58.857546017 +0000 UTC m=+66.101165277 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs") pod "network-metrics-daemon-8wdv9" (UID: "d0178830-4009-4007-9829-a2c0aee48925") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 20:41:26.958709 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:26.958675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:26.958838 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:26.958775    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 20:41:26.958838 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:26.958834    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret podName:6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:58.958819982 +0000 UTC m=+66.202439242 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret") pod "global-pull-secret-syncer-62hnz" (UID: "6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33") : object "kube-system"/"original-pull-secret" not registered
Aug 04 20:41:27.059311 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.059277    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6g28\" (UniqueName: \"kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28\") pod \"network-check-target-bkp8v\" (UID: \"c1997346-3963-43df-b2f3-515df3356344\") " pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:27.059502 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:27.059449    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 20:41:27.059502 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:27.059469    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 20:41:27.059502 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:27.059479    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f6g28 for pod openshift-network-diagnostics/network-check-target-bkp8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:41:27.059637 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:27.059531    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28 podName:c1997346-3963-43df-b2f3-515df3356344 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:59.059516224 +0000 UTC m=+66.303135484 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f6g28" (UniqueName: "kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28") pod "network-check-target-bkp8v" (UID: "c1997346-3963-43df-b2f3-515df3356344") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 20:41:27.237911 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.237841    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:27.238444 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:27.237953    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkp8v" podUID="c1997346-3963-43df-b2f3-515df3356344"
Aug 04 20:41:27.614413 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.614309    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-249.ec2.internal" event="NodeReady"
Aug 04 20:41:27.614575 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.614478    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 04 20:41:27.661253 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.661222    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fg7km"]
Aug 04 20:41:27.665983 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.665963    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f9bch"]
Aug 04 20:41:27.666123 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.666068    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fg7km"
Aug 04 20:41:27.668780 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.668753    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 04 20:41:27.668894 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.668811    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q6crk\""
Aug 04 20:41:27.668894 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.668811    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 04 20:41:27.668992 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.668751    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 04 20:41:27.669458 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.669432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.671610 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.671583    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 04 20:41:27.671686 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.671671    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kcgl6\""
Aug 04 20:41:27.671797 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.671780    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 04 20:41:27.674007 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.673990    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fg7km"]
Aug 04 20:41:27.678139 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.678102    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f9bch"]
Aug 04 20:41:27.764894 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.764858    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fea94516-a7b5-41cc-84c5-10a37f3d021a-config-volume\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.765080 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.764911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3142a84a-6e2c-4b9d-8d2b-a86223d81428-cert\") pod \"ingress-canary-fg7km\" (UID: \"3142a84a-6e2c-4b9d-8d2b-a86223d81428\") " pod="openshift-ingress-canary/ingress-canary-fg7km"
Aug 04 20:41:27.765080 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.764938    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfhmr\" (UniqueName: \"kubernetes.io/projected/3142a84a-6e2c-4b9d-8d2b-a86223d81428-kube-api-access-jfhmr\") pod \"ingress-canary-fg7km\" (UID: \"3142a84a-6e2c-4b9d-8d2b-a86223d81428\") " pod="openshift-ingress-canary/ingress-canary-fg7km"
Aug 04 20:41:27.765080 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.765019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fea94516-a7b5-41cc-84c5-10a37f3d021a-tmp-dir\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.765229 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.765076    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.765229 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.765135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5qfq\" (UniqueName: \"kubernetes.io/projected/fea94516-a7b5-41cc-84c5-10a37f3d021a-kube-api-access-p5qfq\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.866059 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.865970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fea94516-a7b5-41cc-84c5-10a37f3d021a-tmp-dir\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.866059 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.866038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.866282 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.866081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5qfq\" (UniqueName: \"kubernetes.io/projected/fea94516-a7b5-41cc-84c5-10a37f3d021a-kube-api-access-p5qfq\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.866282 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:27.866187    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 20:41:27.866282 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.866205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fea94516-a7b5-41cc-84c5-10a37f3d021a-config-volume\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.866282 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:27.866255    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls podName:fea94516-a7b5-41cc-84c5-10a37f3d021a nodeName:}" failed. No retries permitted until 2026-08-04 20:41:28.366233576 +0000 UTC m=+35.609852852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls") pod "dns-default-f9bch" (UID: "fea94516-a7b5-41cc-84c5-10a37f3d021a") : secret "dns-default-metrics-tls" not found
Aug 04 20:41:27.866519 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.866301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3142a84a-6e2c-4b9d-8d2b-a86223d81428-cert\") pod \"ingress-canary-fg7km\" (UID: \"3142a84a-6e2c-4b9d-8d2b-a86223d81428\") " pod="openshift-ingress-canary/ingress-canary-fg7km"
Aug 04 20:41:27.866519 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.866367    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fea94516-a7b5-41cc-84c5-10a37f3d021a-tmp-dir\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.866519 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.866373    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfhmr\" (UniqueName: \"kubernetes.io/projected/3142a84a-6e2c-4b9d-8d2b-a86223d81428-kube-api-access-jfhmr\") pod \"ingress-canary-fg7km\" (UID: \"3142a84a-6e2c-4b9d-8d2b-a86223d81428\") " pod="openshift-ingress-canary/ingress-canary-fg7km"
Aug 04 20:41:27.866711 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.866687    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fea94516-a7b5-41cc-84c5-10a37f3d021a-config-volume\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.871111 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.870986    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3142a84a-6e2c-4b9d-8d2b-a86223d81428-cert\") pod \"ingress-canary-fg7km\" (UID: \"3142a84a-6e2c-4b9d-8d2b-a86223d81428\") " pod="openshift-ingress-canary/ingress-canary-fg7km"
Aug 04 20:41:27.877738 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.877699    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5qfq\" (UniqueName: \"kubernetes.io/projected/fea94516-a7b5-41cc-84c5-10a37f3d021a-kube-api-access-p5qfq\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:27.877827 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.877741    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfhmr\" (UniqueName: \"kubernetes.io/projected/3142a84a-6e2c-4b9d-8d2b-a86223d81428-kube-api-access-jfhmr\") pod \"ingress-canary-fg7km\" (UID: \"3142a84a-6e2c-4b9d-8d2b-a86223d81428\") " pod="openshift-ingress-canary/ingress-canary-fg7km"
Aug 04 20:41:27.978861 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:27.978824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fg7km"
Aug 04 20:41:28.237676 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:28.237601    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:28.237820 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:28.237786    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:28.240437 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:28.240418    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 20:41:28.240746 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:28.240414    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 20:41:28.240746 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:28.240470    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h8x4x\""
Aug 04 20:41:28.369599 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:28.369565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:28.369754 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:28.369708    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 20:41:28.369819 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:28.369768    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls podName:fea94516-a7b5-41cc-84c5-10a37f3d021a nodeName:}" failed. No retries permitted until 2026-08-04 20:41:29.36975076 +0000 UTC m=+36.613370037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls") pod "dns-default-f9bch" (UID: "fea94516-a7b5-41cc-84c5-10a37f3d021a") : secret "dns-default-metrics-tls" not found
Aug 04 20:41:28.567675 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:28.565909    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fg7km"]
Aug 04 20:41:28.572390 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:41:28.572368    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3142a84a_6e2c_4b9d_8d2b_a86223d81428.slice/crio-f4161a94d2ecd549b935a920416d3a72433f653390e58082fc14150b6e55195d WatchSource:0}: Error finding container f4161a94d2ecd549b935a920416d3a72433f653390e58082fc14150b6e55195d: Status 404 returned error can't find the container with id f4161a94d2ecd549b935a920416d3a72433f653390e58082fc14150b6e55195d
Aug 04 20:41:29.237967 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:29.237931    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:29.241525 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:29.241501    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 20:41:29.242486 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:29.242461    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 20:41:29.242605 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:29.242500    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nn596\""
Aug 04 20:41:29.377577 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:29.377544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:29.377813 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:29.377662    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 20:41:29.377813 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:29.377726    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls podName:fea94516-a7b5-41cc-84c5-10a37f3d021a nodeName:}" failed. No retries permitted until 2026-08-04 20:41:31.377707095 +0000 UTC m=+38.621326355 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls") pod "dns-default-f9bch" (UID: "fea94516-a7b5-41cc-84c5-10a37f3d021a") : secret "dns-default-metrics-tls" not found
Aug 04 20:41:29.406687 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:29.406654    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b89dd0c-99b7-4dce-96c6-3f8e90482915" containerID="f467032e73a7784b09341c9107379465fecaf449587724480ba74ca3328648f0" exitCode=0
Aug 04 20:41:29.406821 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:29.406748    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rpwv5" event={"ID":"9b89dd0c-99b7-4dce-96c6-3f8e90482915","Type":"ContainerDied","Data":"f467032e73a7784b09341c9107379465fecaf449587724480ba74ca3328648f0"}
Aug 04 20:41:29.408032 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:29.407985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fg7km" event={"ID":"3142a84a-6e2c-4b9d-8d2b-a86223d81428","Type":"ContainerStarted","Data":"f4161a94d2ecd549b935a920416d3a72433f653390e58082fc14150b6e55195d"}
Aug 04 20:41:30.414288 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:30.414110    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b89dd0c-99b7-4dce-96c6-3f8e90482915" containerID="dd170451b7267a6b47849307214291d369e767bfb7306507e801e77196774872" exitCode=0
Aug 04 20:41:30.414671 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:30.414183    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rpwv5" event={"ID":"9b89dd0c-99b7-4dce-96c6-3f8e90482915","Type":"ContainerDied","Data":"dd170451b7267a6b47849307214291d369e767bfb7306507e801e77196774872"}
Aug 04 20:41:31.392104 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:31.392078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:31.392277 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:31.392243    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 20:41:31.392373 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:31.392337    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls podName:fea94516-a7b5-41cc-84c5-10a37f3d021a nodeName:}" failed. No retries permitted until 2026-08-04 20:41:35.3923015 +0000 UTC m=+42.635920782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls") pod "dns-default-f9bch" (UID: "fea94516-a7b5-41cc-84c5-10a37f3d021a") : secret "dns-default-metrics-tls" not found
Aug 04 20:41:31.419661 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:31.419627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rpwv5" event={"ID":"9b89dd0c-99b7-4dce-96c6-3f8e90482915","Type":"ContainerStarted","Data":"269475eb97a1470f9332547c683d573529de7ea72067b82aea2ca8ff0675a0fa"}
Aug 04 20:41:31.420884 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:31.420865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fg7km" event={"ID":"3142a84a-6e2c-4b9d-8d2b-a86223d81428","Type":"ContainerStarted","Data":"91c11fddafbcf84b8afd4736c101b0b88e9401e442c8f70df124f4bc9aecd8f7"}
Aug 04 20:41:31.449050 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:31.448998    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rpwv5" podStartSLOduration=8.635757338 podStartE2EDuration="38.448982007s" totalImagesPullingTime="29.813224669s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:31.448502427 +0000 UTC m=+38.692121715" watchObservedRunningTime="2026-08-04 20:41:31.448982007 +0000 UTC m=+38.692601289"
Aug 04 20:41:31.466342 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:31.466282    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fg7km" podStartSLOduration=2.3447495910000002 podStartE2EDuration="4.46626776s" totalImagesPullingTime="2.121518169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:41:31.465740591 +0000 UTC m=+38.709359884" watchObservedRunningTime="2026-08-04 20:41:31.46626776 +0000 UTC m=+38.709887023"
Aug 04 20:41:35.419960 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:35.419932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:35.420344 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:35.420060    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 20:41:35.420344 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:35.420110    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls podName:fea94516-a7b5-41cc-84c5-10a37f3d021a nodeName:}" failed. No retries permitted until 2026-08-04 20:41:43.420096772 +0000 UTC m=+50.663716033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls") pod "dns-default-f9bch" (UID: "fea94516-a7b5-41cc-84c5-10a37f3d021a") : secret "dns-default-metrics-tls" not found
Aug 04 20:41:43.471263 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:43.471232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:43.471708 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:43.471400    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 20:41:43.471708 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:43.471491    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls podName:fea94516-a7b5-41cc-84c5-10a37f3d021a nodeName:}" failed. No retries permitted until 2026-08-04 20:41:59.471474382 +0000 UTC m=+66.715093643 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls") pod "dns-default-f9bch" (UID: "fea94516-a7b5-41cc-84c5-10a37f3d021a") : secret "dns-default-metrics-tls" not found
Aug 04 20:41:53.398926 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:53.398896    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8rgcs"
Aug 04 20:41:58.240734 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.240702    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"]
Aug 04 20:41:58.243246 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.243225    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:41:58.248407 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.248381    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dszrb\""
Aug 04 20:41:58.248531 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.248420    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 04 20:41:58.248531 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.248420    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Aug 04 20:41:58.248531 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.248448    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 04 20:41:58.248531 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.248394    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Aug 04 20:41:58.264480 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.264455    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"]
Aug 04 20:41:58.266831 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.266807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:41:58.266918 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.266854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:41:58.266972 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.266954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4zj4\" (UniqueName: \"kubernetes.io/projected/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-kube-api-access-j4zj4\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:41:58.353158 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.353135    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6765f794fb-9thrr"]
Aug 04 20:41:58.354592 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.354579    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.361167 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.361148    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Aug 04 20:41:58.361679 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.361665    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Aug 04 20:41:58.361747 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.361683    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Aug 04 20:41:58.367139 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367127    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Aug 04 20:41:58.367234 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-default-certificate\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.367302 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4zj4\" (UniqueName: \"kubernetes.io/projected/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-kube-api-access-j4zj4\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:41:58.367370 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367345    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 04 20:41:58.367370 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367359    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.367464 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367390    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tbxr\" (UniqueName: \"kubernetes.io/projected/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-kube-api-access-4tbxr\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.367464 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367445    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:41:58.367464 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367457    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bhd2j\""
Aug 04 20:41:58.367596 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.367596 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:41:58.367596 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.367532    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 20:41:58.367596 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-stats-auth\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.367596 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.367571    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 04 20:41:58.367596 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.367588    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls podName:3dbdac82-ac1f-49c9-9f64-98c6f63e8076 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:58.867567749 +0000 UTC m=+66.111187032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7cwsh" (UID: "3dbdac82-ac1f-49c9-9f64-98c6f63e8076") : secret "cluster-monitoring-operator-tls" not found
Aug 04 20:41:58.368107 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.368089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:41:58.382442 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.382414    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6765f794fb-9thrr"]
Aug 04 20:41:58.389010 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.388989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4zj4\" (UniqueName: \"kubernetes.io/projected/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-kube-api-access-j4zj4\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:41:58.468719 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.468693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tbxr\" (UniqueName: \"kubernetes.io/projected/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-kube-api-access-4tbxr\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.468830 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.468740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.468830 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.468764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-stats-auth\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.468830 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.468780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-default-certificate\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.468830 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.468826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.469027 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.468874    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:58.968856569 +0000 UTC m=+66.212475849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : configmap references non-existent config key: service-ca.crt
Aug 04 20:41:58.469027 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.468903    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 20:41:58.469027 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.468955    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:58.968940637 +0000 UTC m=+66.212559915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : secret "router-metrics-certs-default" not found
Aug 04 20:41:58.471069 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.471048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-default-certificate\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.471144 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.471086    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-stats-auth\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.491404 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.491359    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tbxr\" (UniqueName: \"kubernetes.io/projected/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-kube-api-access-4tbxr\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.870661 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.870635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:41:58.870784 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.870696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:41:58.870829 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.870785    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 20:41:58.870862 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.870839    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls podName:3dbdac82-ac1f-49c9-9f64-98c6f63e8076 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:59.870827213 +0000 UTC m=+67.114446473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7cwsh" (UID: "3dbdac82-ac1f-49c9-9f64-98c6f63e8076") : secret "cluster-monitoring-operator-tls" not found
Aug 04 20:41:58.873640 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.873624    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 20:41:58.881099 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.881082    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 04 20:41:58.881179 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.881141    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs podName:d0178830-4009-4007-9829-a2c0aee48925 nodeName:}" failed. No retries permitted until 2026-08-04 20:43:02.881124295 +0000 UTC m=+130.124743556 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs") pod "network-metrics-daemon-8wdv9" (UID: "d0178830-4009-4007-9829-a2c0aee48925") : secret "metrics-daemon-secret" not found
Aug 04 20:41:58.971255 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.971233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.971338 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.971264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:58.971338 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.971288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:58.971442 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.971390    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 20:41:58.971442 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.971394    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:59.971383848 +0000 UTC m=+67.215003109 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : configmap references non-existent config key: service-ca.crt
Aug 04 20:41:58.971522 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:58.971453    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:41:59.971437169 +0000 UTC m=+67.215056435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : secret "router-metrics-certs-default" not found
Aug 04 20:41:58.974037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.974020    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 20:41:58.983526 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:58.983502    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33-original-pull-secret\") pod \"global-pull-secret-syncer-62hnz\" (UID: \"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33\") " pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:59.071994 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.071974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6g28\" (UniqueName: \"kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28\") pod \"network-check-target-bkp8v\" (UID: \"c1997346-3963-43df-b2f3-515df3356344\") " pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:59.074815 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.074793    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 20:41:59.085561 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.085544    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 20:41:59.095556 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.095535    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6g28\" (UniqueName: \"kubernetes.io/projected/c1997346-3963-43df-b2f3-515df3356344-kube-api-access-f6g28\") pod \"network-check-target-bkp8v\" (UID: \"c1997346-3963-43df-b2f3-515df3356344\") " pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:59.149446 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.149405    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62hnz"
Aug 04 20:41:59.252537 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.252516    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nn596\""
Aug 04 20:41:59.260555 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.260537    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:41:59.284330 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.284285    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-62hnz"]
Aug 04 20:41:59.287188 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:41:59.287157    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a2ac5f5_4e0f_4ac4_92cf_2281241d4e33.slice/crio-29fa9b1d92e27df854487a60b27acc171f0b13c01bc99c45a71279241c161b20 WatchSource:0}: Error finding container 29fa9b1d92e27df854487a60b27acc171f0b13c01bc99c45a71279241c161b20: Status 404 returned error can't find the container with id 29fa9b1d92e27df854487a60b27acc171f0b13c01bc99c45a71279241c161b20
Aug 04 20:41:59.392713 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.392687    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bkp8v"]
Aug 04 20:41:59.396536 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:41:59.396513    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1997346_3963_43df_b2f3_515df3356344.slice/crio-bf38a12ae7b485de76ba1b3b5b608e9f74a8206888d14e5b6c3a608e31a2829c WatchSource:0}: Error finding container bf38a12ae7b485de76ba1b3b5b608e9f74a8206888d14e5b6c3a608e31a2829c: Status 404 returned error can't find the container with id bf38a12ae7b485de76ba1b3b5b608e9f74a8206888d14e5b6c3a608e31a2829c
Aug 04 20:41:59.476112 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.476087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:41:59.476248 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:59.476234    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 20:41:59.476304 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:59.476288    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls podName:fea94516-a7b5-41cc-84c5-10a37f3d021a nodeName:}" failed. No retries permitted until 2026-08-04 20:42:31.476270356 +0000 UTC m=+98.719889617 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls") pod "dns-default-f9bch" (UID: "fea94516-a7b5-41cc-84c5-10a37f3d021a") : secret "dns-default-metrics-tls" not found
Aug 04 20:41:59.484550 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.484521    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-62hnz" event={"ID":"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33","Type":"ContainerStarted","Data":"29fa9b1d92e27df854487a60b27acc171f0b13c01bc99c45a71279241c161b20"}
Aug 04 20:41:59.485470 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.485448    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bkp8v" event={"ID":"c1997346-3963-43df-b2f3-515df3356344","Type":"ContainerStarted","Data":"bf38a12ae7b485de76ba1b3b5b608e9f74a8206888d14e5b6c3a608e31a2829c"}
Aug 04 20:41:59.879614 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.879587    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:41:59.879705 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:59.879696    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 20:41:59.879749 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:59.879739    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls podName:3dbdac82-ac1f-49c9-9f64-98c6f63e8076 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:01.87972662 +0000 UTC m=+69.123345881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7cwsh" (UID: "3dbdac82-ac1f-49c9-9f64-98c6f63e8076") : secret "cluster-monitoring-operator-tls" not found
Aug 04 20:41:59.980862 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.980838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:59.980959 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:41:59.980920    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:41:59.981031 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:59.981007    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:01.980985058 +0000 UTC m=+69.224604333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : configmap references non-existent config key: service-ca.crt
Aug 04 20:41:59.981031 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:59.981009    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 20:41:59.981221 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:41:59.981060    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:01.9810485 +0000 UTC m=+69.224667765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : secret "router-metrics-certs-default" not found
Aug 04 20:42:01.894581 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.894535    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:42:01.895034 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:01.894709    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 20:42:01.895034 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:01.894791    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls podName:3dbdac82-ac1f-49c9-9f64-98c6f63e8076 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:05.894771573 +0000 UTC m=+73.138390833 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7cwsh" (UID: "3dbdac82-ac1f-49c9-9f64-98c6f63e8076") : secret "cluster-monitoring-operator-tls" not found
Aug 04 20:42:01.921923 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.921897    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-f872z"]
Aug 04 20:42:01.925086 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.925067    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:01.927853 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.927827    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 04 20:42:01.928102 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.928081    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 04 20:42:01.928193 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.928099    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d7shv\""
Aug 04 20:42:01.928974 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.928953    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 04 20:42:01.929108 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.928963    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 04 20:42:01.942831 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.942813    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f872z"]
Aug 04 20:42:01.995922 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.995881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:01.996076 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.995936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:01.996076 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.995967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-crio-socket\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:01.996076 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.995989    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-data-volume\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:01.996076 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:01.996010    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 20:42:01.996076 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.996021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:01.996372 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:01.996086    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:05.996055189 +0000 UTC m=+73.239674452 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : secret "router-metrics-certs-default" not found
Aug 04 20:42:01.996372 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:01.996106    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:05.996096667 +0000 UTC m=+73.239715930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : configmap references non-existent config key: service-ca.crt
Aug 04 20:42:01.996372 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.996139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:01.996372 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:01.996167    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btrmf\" (UniqueName: \"kubernetes.io/projected/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-api-access-btrmf\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:02.096824 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:02.096794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-crio-socket\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:02.096824 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:02.096824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-data-volume\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:02.097075 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:02.096859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:02.097075 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:02.096897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:02.097075 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:02.096925    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btrmf\" (UniqueName: \"kubernetes.io/projected/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-api-access-btrmf\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:02.097075 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:02.096932    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-crio-socket\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:02.097075 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:02.097020    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 20:42:02.097075 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:02.097028    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:02.097075 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:02.097079    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm podName:db41f9ff-f898-4cda-b1b0-2bc2dba136ba nodeName:}" failed. No retries permitted until 2026-08-04 20:42:02.597058638 +0000 UTC m=+69.840677902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f872z" (UID: "db41f9ff-f898-4cda-b1b0-2bc2dba136ba") : configmap "kube-rbac-proxy" not found
Aug 04 20:42:02.097345 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:02.097096    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls podName:db41f9ff-f898-4cda-b1b0-2bc2dba136ba nodeName:}" failed. No retries permitted until 2026-08-04 20:42:02.597086566 +0000 UTC m=+69.840705831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f872z" (UID: "db41f9ff-f898-4cda-b1b0-2bc2dba136ba") : secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:02.097345 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:02.097192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-data-volume\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:02.111901 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:02.111854    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btrmf\" (UniqueName: \"kubernetes.io/projected/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-api-access-btrmf\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:02.600817 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:02.600783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:02.601001 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:02.600920    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 20:42:02.601001 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:02.600939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:02.601001 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:02.600986    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm podName:db41f9ff-f898-4cda-b1b0-2bc2dba136ba nodeName:}" failed. No retries permitted until 2026-08-04 20:42:03.600969936 +0000 UTC m=+70.844589196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f872z" (UID: "db41f9ff-f898-4cda-b1b0-2bc2dba136ba") : configmap "kube-rbac-proxy" not found
Aug 04 20:42:02.601173 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:02.601054    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:02.601173 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:02.601103    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls podName:db41f9ff-f898-4cda-b1b0-2bc2dba136ba nodeName:}" failed. No retries permitted until 2026-08-04 20:42:03.60108736 +0000 UTC m=+70.844706626 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f872z" (UID: "db41f9ff-f898-4cda-b1b0-2bc2dba136ba") : secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:03.324363 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.324310    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz"]
Aug 04 20:42:03.326907 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.326358    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz"
Aug 04 20:42:03.329195 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.329175    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Aug 04 20:42:03.330208 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.330185    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Aug 04 20:42:03.330208 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.330203    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-zjmkt\""
Aug 04 20:42:03.330410 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.330308    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Aug 04 20:42:03.330458 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.330423    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Aug 04 20:42:03.343153 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.343130    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz"]
Aug 04 20:42:03.410817 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.410774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/abc76fe7-72c5-41f6-8c30-7a912cd317e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz\" (UID: \"abc76fe7-72c5-41f6-8c30-7a912cd317e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz"
Aug 04 20:42:03.410817 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.410818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mh4c\" (UniqueName: \"kubernetes.io/projected/abc76fe7-72c5-41f6-8c30-7a912cd317e4-kube-api-access-8mh4c\") pod \"managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz\" (UID: \"abc76fe7-72c5-41f6-8c30-7a912cd317e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz"
Aug 04 20:42:03.511466 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.511437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/abc76fe7-72c5-41f6-8c30-7a912cd317e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz\" (UID: \"abc76fe7-72c5-41f6-8c30-7a912cd317e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz"
Aug 04 20:42:03.511641 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.511472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mh4c\" (UniqueName: \"kubernetes.io/projected/abc76fe7-72c5-41f6-8c30-7a912cd317e4-kube-api-access-8mh4c\") pod \"managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz\" (UID: \"abc76fe7-72c5-41f6-8c30-7a912cd317e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz"
Aug 04 20:42:03.514562 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.514539    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/abc76fe7-72c5-41f6-8c30-7a912cd317e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz\" (UID: \"abc76fe7-72c5-41f6-8c30-7a912cd317e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz"
Aug 04 20:42:03.520247 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.520227    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mh4c\" (UniqueName: \"kubernetes.io/projected/abc76fe7-72c5-41f6-8c30-7a912cd317e4-kube-api-access-8mh4c\") pod \"managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz\" (UID: \"abc76fe7-72c5-41f6-8c30-7a912cd317e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz"
Aug 04 20:42:03.612651 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.612567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:03.612813 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.612677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:03.612813 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:03.612696    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 20:42:03.612813 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:03.612769    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm podName:db41f9ff-f898-4cda-b1b0-2bc2dba136ba nodeName:}" failed. No retries permitted until 2026-08-04 20:42:05.61275126 +0000 UTC m=+72.856370522 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f872z" (UID: "db41f9ff-f898-4cda-b1b0-2bc2dba136ba") : configmap "kube-rbac-proxy" not found
Aug 04 20:42:03.612813 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:03.612781    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:03.613027 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:03.612828    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls podName:db41f9ff-f898-4cda-b1b0-2bc2dba136ba nodeName:}" failed. No retries permitted until 2026-08-04 20:42:05.612812791 +0000 UTC m=+72.856432054 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f872z" (UID: "db41f9ff-f898-4cda-b1b0-2bc2dba136ba") : secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:03.647251 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.647226    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz"
Aug 04 20:42:03.855039 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:03.855016    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kblfw_62b003b3-9dd3-4f30-b0ac-c88b3003e065/dns-node-resolver/0.log"
Aug 04 20:42:04.052524 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:04.052477    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz"]
Aug 04 20:42:04.055796 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:42:04.055771    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabc76fe7_72c5_41f6_8c30_7a912cd317e4.slice/crio-a578a51549151c0db0684b37151381e6537f1e2e9beb1cac00d5f7f6432d9f89 WatchSource:0}: Error finding container a578a51549151c0db0684b37151381e6537f1e2e9beb1cac00d5f7f6432d9f89: Status 404 returned error can't find the container with id a578a51549151c0db0684b37151381e6537f1e2e9beb1cac00d5f7f6432d9f89
Aug 04 20:42:04.499615 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:04.499578    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bkp8v" event={"ID":"c1997346-3963-43df-b2f3-515df3356344","Type":"ContainerStarted","Data":"7cbab5d1f5091573f8d5674b55563f4ab99c68845812b7d1dc1ce75096da351f"}
Aug 04 20:42:04.500051 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:04.499643    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:42:04.500590 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:04.500571    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz" event={"ID":"abc76fe7-72c5-41f6-8c30-7a912cd317e4","Type":"ContainerStarted","Data":"a578a51549151c0db0684b37151381e6537f1e2e9beb1cac00d5f7f6432d9f89"}
Aug 04 20:42:04.501721 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:04.501702    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-62hnz" event={"ID":"6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33","Type":"ContainerStarted","Data":"c05872981aa1d2ca3044f2e28e400d187e2670f07c04c0ad1bfd3670fc0ad309"}
Aug 04 20:42:04.526416 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:04.526378    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bkp8v" podStartSLOduration=67.017225474 podStartE2EDuration="1m11.526365188s" totalImagesPullingTime="4.509139714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:04.525728339 +0000 UTC m=+71.769347621" watchObservedRunningTime="2026-08-04 20:42:04.526365188 +0000 UTC m=+71.769984448"
Aug 04 20:42:04.548715 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:04.548680    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-62hnz" podStartSLOduration=66.927043415 podStartE2EDuration="1m11.548669025s" totalImagesPullingTime="4.62162561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:04.547602051 +0000 UTC m=+71.791221332" watchObservedRunningTime="2026-08-04 20:42:04.548669025 +0000 UTC m=+71.792288307"
Aug 04 20:42:04.642046 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:04.642026    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w7fs7_f558bf91-5880-4917-9e1e-c78ae5493458/node-ca/0.log"
Aug 04 20:42:05.629039 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:05.629005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:05.629039 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:05.629048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:05.629495 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:05.629134    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 20:42:05.629495 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:05.629160    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:05.629495 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:05.629201    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm podName:db41f9ff-f898-4cda-b1b0-2bc2dba136ba nodeName:}" failed. No retries permitted until 2026-08-04 20:42:09.629181651 +0000 UTC m=+76.872800932 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f872z" (UID: "db41f9ff-f898-4cda-b1b0-2bc2dba136ba") : configmap "kube-rbac-proxy" not found
Aug 04 20:42:05.629495 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:05.629221    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls podName:db41f9ff-f898-4cda-b1b0-2bc2dba136ba nodeName:}" failed. No retries permitted until 2026-08-04 20:42:09.629208175 +0000 UTC m=+76.872827439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f872z" (UID: "db41f9ff-f898-4cda-b1b0-2bc2dba136ba") : secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:05.642571 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:05.642549    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fg7km_3142a84a-6e2c-4b9d-8d2b-a86223d81428/serve-healthcheck-canary/0.log"
Aug 04 20:42:05.931342 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:05.931241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:42:05.931489 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:05.931440    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 20:42:05.931527 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:05.931521    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls podName:3dbdac82-ac1f-49c9-9f64-98c6f63e8076 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:13.931499809 +0000 UTC m=+81.175119097 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7cwsh" (UID: "3dbdac82-ac1f-49c9-9f64-98c6f63e8076") : secret "cluster-monitoring-operator-tls" not found
Aug 04 20:42:06.032169 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.032136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:06.032375 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.032205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:06.032596 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:06.032338    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 20:42:06.032801 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:06.032787    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:14.032750902 +0000 UTC m=+81.276370176 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : secret "router-metrics-certs-default" not found
Aug 04 20:42:06.032867 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:06.032823    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:14.032812432 +0000 UTC m=+81.276431694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : configmap references non-existent config key: service-ca.crt
Aug 04 20:42:06.228254 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.228184    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b757f955-dkhmx"]
Aug 04 20:42:06.230196 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.230175    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.233784 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.233757    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 04 20:42:06.233784 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.233773    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 04 20:42:06.234093 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.234070    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sm2np\""
Aug 04 20:42:06.237384 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.237364    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 04 20:42:06.243937 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.243921    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 04 20:42:06.251463 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.251443    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b757f955-dkhmx"]
Aug 04 20:42:06.334666 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.334625    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.334820 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.334693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87z2q\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-kube-api-access-87z2q\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.334896 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.334811    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-registry-certificates\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.334896 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.334854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-installation-pull-secrets\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.334896 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.334884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-bound-sa-token\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.335045 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.334937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d81429a-932e-4050-8b72-4efb1f021150-ca-trust-extracted\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.335045 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.334991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-trusted-ca\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.335045 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.335032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-image-registry-private-configuration\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.435448 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.435412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-bound-sa-token\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.435633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.435460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d81429a-932e-4050-8b72-4efb1f021150-ca-trust-extracted\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.435633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.435504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-trusted-ca\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.435633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.435528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-image-registry-private-configuration\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.435633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.435574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.435633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.435628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87z2q\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-kube-api-access-87z2q\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.435877 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.435700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-registry-certificates\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.435877 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.435743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-installation-pull-secrets\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.435877 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:06.435765    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 20:42:06.435877 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:06.435792    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b757f955-dkhmx: secret "image-registry-tls" not found
Aug 04 20:42:06.435877 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:06.435867    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls podName:9d81429a-932e-4050-8b72-4efb1f021150 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:06.935839091 +0000 UTC m=+74.179458367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls") pod "image-registry-7b757f955-dkhmx" (UID: "9d81429a-932e-4050-8b72-4efb1f021150") : secret "image-registry-tls" not found
Aug 04 20:42:06.436045 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.435987    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d81429a-932e-4050-8b72-4efb1f021150-ca-trust-extracted\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.436737 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.436687    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-registry-certificates\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.437282 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.437233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-trusted-ca\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.438872 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.438852    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-installation-pull-secrets\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.438872 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.438866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-image-registry-private-configuration\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.451787 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.451756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87z2q\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-kube-api-access-87z2q\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.451906 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.451880    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-bound-sa-token\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.939005 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:06.938977    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:06.939378 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:06.939130    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 20:42:06.939378 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:06.939150    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b757f955-dkhmx: secret "image-registry-tls" not found
Aug 04 20:42:06.939378 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:06.939202    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls podName:9d81429a-932e-4050-8b72-4efb1f021150 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:07.939187062 +0000 UTC m=+75.182806323 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls") pod "image-registry-7b757f955-dkhmx" (UID: "9d81429a-932e-4050-8b72-4efb1f021150") : secret "image-registry-tls" not found
Aug 04 20:42:07.510498 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:07.510465    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz" event={"ID":"abc76fe7-72c5-41f6-8c30-7a912cd317e4","Type":"ContainerStarted","Data":"cb584a106b6e1f89c923ff96fb7c113c6c7d49211b0ea9c8d9085697c348af1b"}
Aug 04 20:42:07.528133 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:07.528097    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b5cb6587b-7l6fz" podStartSLOduration=2.105907515 podStartE2EDuration="4.528084202s" totalImagesPullingTime="2.422176687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:07.527422797 +0000 UTC m=+74.771042079" watchObservedRunningTime="2026-08-04 20:42:07.528084202 +0000 UTC m=+74.771703483"
Aug 04 20:42:07.946752 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:07.946711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:07.947065 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:07.946848    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 20:42:07.947065 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:07.946865    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b757f955-dkhmx: secret "image-registry-tls" not found
Aug 04 20:42:07.947065 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:07.946914    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls podName:9d81429a-932e-4050-8b72-4efb1f021150 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:09.946899761 +0000 UTC m=+77.190519025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls") pod "image-registry-7b757f955-dkhmx" (UID: "9d81429a-932e-4050-8b72-4efb1f021150") : secret "image-registry-tls" not found
Aug 04 20:42:09.659665 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:09.659626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:09.659665 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:09.659669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:09.660082 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:09.659760    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:09.660082 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:09.659812    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls podName:db41f9ff-f898-4cda-b1b0-2bc2dba136ba nodeName:}" failed. No retries permitted until 2026-08-04 20:42:17.659798568 +0000 UTC m=+84.903417828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f872z" (UID: "db41f9ff-f898-4cda-b1b0-2bc2dba136ba") : secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:09.660082 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:09.659811    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 20:42:09.660082 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:09.659858    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm podName:db41f9ff-f898-4cda-b1b0-2bc2dba136ba nodeName:}" failed. No retries permitted until 2026-08-04 20:42:17.65984505 +0000 UTC m=+84.903464311 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f872z" (UID: "db41f9ff-f898-4cda-b1b0-2bc2dba136ba") : configmap "kube-rbac-proxy" not found
Aug 04 20:42:09.962074 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:09.962016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:09.962278 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:09.962165    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 20:42:09.962278 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:09.962181    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b757f955-dkhmx: secret "image-registry-tls" not found
Aug 04 20:42:09.962278 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:09.962237    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls podName:9d81429a-932e-4050-8b72-4efb1f021150 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:13.962220248 +0000 UTC m=+81.205839522 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls") pod "image-registry-7b757f955-dkhmx" (UID: "9d81429a-932e-4050-8b72-4efb1f021150") : secret "image-registry-tls" not found
Aug 04 20:42:12.841086 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:12.841051    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n"]
Aug 04 20:42:12.843936 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:12.843921    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n"
Aug 04 20:42:12.847843 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:12.847820    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Aug 04 20:42:12.847843 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:12.847821    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Aug 04 20:42:12.848039 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:12.847868    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5ktrx\""
Aug 04 20:42:12.855897 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:12.855877    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n"]
Aug 04 20:42:12.881643 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:12.881618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrg54\" (UniqueName: \"kubernetes.io/projected/00bac56d-cfc6-441d-9a61-126ff8c60642-kube-api-access-zrg54\") pod \"migrator-68bc745d87-ckq4n\" (UID: \"00bac56d-cfc6-441d-9a61-126ff8c60642\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n"
Aug 04 20:42:12.982057 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:12.982033    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrg54\" (UniqueName: \"kubernetes.io/projected/00bac56d-cfc6-441d-9a61-126ff8c60642-kube-api-access-zrg54\") pod \"migrator-68bc745d87-ckq4n\" (UID: \"00bac56d-cfc6-441d-9a61-126ff8c60642\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n"
Aug 04 20:42:12.998868 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:12.998846    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrg54\" (UniqueName: \"kubernetes.io/projected/00bac56d-cfc6-441d-9a61-126ff8c60642-kube-api-access-zrg54\") pod \"migrator-68bc745d87-ckq4n\" (UID: \"00bac56d-cfc6-441d-9a61-126ff8c60642\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n"
Aug 04 20:42:13.151600 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:13.151580    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n"
Aug 04 20:42:13.277274 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:13.277250    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n"]
Aug 04 20:42:13.281259 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:42:13.281232    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00bac56d_cfc6_441d_9a61_126ff8c60642.slice/crio-81a3320d0010115bf8de189959731894c5b32b355f8bb52258eb7c833f7cac7d WatchSource:0}: Error finding container 81a3320d0010115bf8de189959731894c5b32b355f8bb52258eb7c833f7cac7d: Status 404 returned error can't find the container with id 81a3320d0010115bf8de189959731894c5b32b355f8bb52258eb7c833f7cac7d
Aug 04 20:42:13.525732 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:13.525677    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n" event={"ID":"00bac56d-cfc6-441d-9a61-126ff8c60642","Type":"ContainerStarted","Data":"81a3320d0010115bf8de189959731894c5b32b355f8bb52258eb7c833f7cac7d"}
Aug 04 20:42:13.988283 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:13.988248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:42:13.988709 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:13.988353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:13.988709 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:13.988437    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 20:42:13.988709 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:13.988464    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 20:42:13.988709 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:13.988475    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b757f955-dkhmx: secret "image-registry-tls" not found
Aug 04 20:42:13.988709 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:13.988518    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls podName:3dbdac82-ac1f-49c9-9f64-98c6f63e8076 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:29.988487852 +0000 UTC m=+97.232107116 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-7cwsh" (UID: "3dbdac82-ac1f-49c9-9f64-98c6f63e8076") : secret "cluster-monitoring-operator-tls" not found
Aug 04 20:42:13.988709 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:13.988534    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls podName:9d81429a-932e-4050-8b72-4efb1f021150 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:21.98852781 +0000 UTC m=+89.232147070 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls") pod "image-registry-7b757f955-dkhmx" (UID: "9d81429a-932e-4050-8b72-4efb1f021150") : secret "image-registry-tls" not found
Aug 04 20:42:14.089661 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:14.089619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:14.089812 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:14.089723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:14.089812 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:14.089780    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 20:42:14.089911 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:14.089856    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:30.089834843 +0000 UTC m=+97.333454107 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : secret "router-metrics-certs-default" not found
Aug 04 20:42:14.089911 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:14.089873    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle podName:6bea9a8c-2700-49e3-8d87-3456d6d2c3e9 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:30.089865641 +0000 UTC m=+97.333484903 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle") pod "router-default-6765f794fb-9thrr" (UID: "6bea9a8c-2700-49e3-8d87-3456d6d2c3e9") : configmap references non-existent config key: service-ca.crt
Aug 04 20:42:14.530586 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:14.530530    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n" event={"ID":"00bac56d-cfc6-441d-9a61-126ff8c60642","Type":"ContainerStarted","Data":"a1b63a970729e6c93022757e84360026ba5dc95a8f3942dc97a5638f34024225"}
Aug 04 20:42:14.530678 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:14.530621    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n" event={"ID":"00bac56d-cfc6-441d-9a61-126ff8c60642","Type":"ContainerStarted","Data":"4d3878524554c047a85c86369bf4f64365a09026d539932834b2412b01ae55a7"}
Aug 04 20:42:14.547336 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:14.547220    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-ckq4n" podStartSLOduration=1.554944693 podStartE2EDuration="2.5472077s" totalImagesPullingTime="992.263007ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:14.546808958 +0000 UTC m=+81.790428239" watchObservedRunningTime="2026-08-04 20:42:14.5472077 +0000 UTC m=+81.790827034"
Aug 04 20:42:17.714983 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:17.714945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:17.714983 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:17.714986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:17.715421 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:17.715094    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:17.715421 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:17.715159    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls podName:db41f9ff-f898-4cda-b1b0-2bc2dba136ba nodeName:}" failed. No retries permitted until 2026-08-04 20:42:33.715143216 +0000 UTC m=+100.958762493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f872z" (UID: "db41f9ff-f898-4cda-b1b0-2bc2dba136ba") : secret "insights-runtime-extractor-tls" not found
Aug 04 20:42:17.715568 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:17.715551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:22.044876 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:22.044843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:22.047087 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:22.047065    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls\") pod \"image-registry-7b757f955-dkhmx\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") " pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:22.141044 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:22.141019    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:22.265895 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:22.265747    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b757f955-dkhmx"]
Aug 04 20:42:22.268369 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:42:22.268336    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d81429a_932e_4050_8b72_4efb1f021150.slice/crio-77b09d787c14dd4720dab8d2ebbbe898346164ee1e0cd33e707e0a805917d1b8 WatchSource:0}: Error finding container 77b09d787c14dd4720dab8d2ebbbe898346164ee1e0cd33e707e0a805917d1b8: Status 404 returned error can't find the container with id 77b09d787c14dd4720dab8d2ebbbe898346164ee1e0cd33e707e0a805917d1b8
Aug 04 20:42:22.556933 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:22.556860    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b757f955-dkhmx" event={"ID":"9d81429a-932e-4050-8b72-4efb1f021150","Type":"ContainerStarted","Data":"4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39"}
Aug 04 20:42:22.556933 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:22.556910    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b757f955-dkhmx" event={"ID":"9d81429a-932e-4050-8b72-4efb1f021150","Type":"ContainerStarted","Data":"77b09d787c14dd4720dab8d2ebbbe898346164ee1e0cd33e707e0a805917d1b8"}
Aug 04 20:42:22.557105 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:22.556980    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:42:22.578432 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:22.578380    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b757f955-dkhmx" podStartSLOduration=16.578365857 podStartE2EDuration="16.578365857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:22.576970028 +0000 UTC m=+89.820589311" watchObservedRunningTime="2026-08-04 20:42:22.578365857 +0000 UTC m=+89.821985138"
Aug 04 20:42:29.995558 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:29.995524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:42:29.997999 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:29.997975    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dbdac82-ac1f-49c9-9f64-98c6f63e8076-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-7cwsh\" (UID: \"3dbdac82-ac1f-49c9-9f64-98c6f63e8076\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:42:30.051378 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.051358    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"
Aug 04 20:42:30.096460 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.096426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:30.096633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.096518    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:30.097333 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.097285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-service-ca-bundle\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:30.099696 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.099671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bea9a8c-2700-49e3-8d87-3456d6d2c3e9-metrics-certs\") pod \"router-default-6765f794fb-9thrr\" (UID: \"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9\") " pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:30.162246 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.162225    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:30.171884 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.171863    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh"]
Aug 04 20:42:30.174644 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:42:30.174605    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dbdac82_ac1f_49c9_9f64_98c6f63e8076.slice/crio-baa97bbf728a536b857b5fc395c3432202f153f31d5dfd517e64370f59b371e9 WatchSource:0}: Error finding container baa97bbf728a536b857b5fc395c3432202f153f31d5dfd517e64370f59b371e9: Status 404 returned error can't find the container with id baa97bbf728a536b857b5fc395c3432202f153f31d5dfd517e64370f59b371e9
Aug 04 20:42:30.284335 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.284253    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6765f794fb-9thrr"]
Aug 04 20:42:30.286860 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:42:30.286839    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bea9a8c_2700_49e3_8d87_3456d6d2c3e9.slice/crio-b7978767ecb6b588e7d218c2b3d62de77765e3c1205c3f8221b53e4508dfc32e WatchSource:0}: Error finding container b7978767ecb6b588e7d218c2b3d62de77765e3c1205c3f8221b53e4508dfc32e: Status 404 returned error can't find the container with id b7978767ecb6b588e7d218c2b3d62de77765e3c1205c3f8221b53e4508dfc32e
Aug 04 20:42:30.580697 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.580629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh" event={"ID":"3dbdac82-ac1f-49c9-9f64-98c6f63e8076","Type":"ContainerStarted","Data":"baa97bbf728a536b857b5fc395c3432202f153f31d5dfd517e64370f59b371e9"}
Aug 04 20:42:30.582055 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.582029    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6765f794fb-9thrr" event={"ID":"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9","Type":"ContainerStarted","Data":"b1409184744d6b00063580e246d788129b46cd16c927fc9e9a77a6f6613c21a1"}
Aug 04 20:42:30.582174 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.582067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6765f794fb-9thrr" event={"ID":"6bea9a8c-2700-49e3-8d87-3456d6d2c3e9","Type":"ContainerStarted","Data":"b7978767ecb6b588e7d218c2b3d62de77765e3c1205c3f8221b53e4508dfc32e"}
Aug 04 20:42:30.605651 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:30.605612    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6765f794fb-9thrr" podStartSLOduration=32.605597579 podStartE2EDuration="32.605597579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:41:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:30.605043581 +0000 UTC m=+97.848662862" watchObservedRunningTime="2026-08-04 20:42:30.605597579 +0000 UTC m=+97.849216860"
Aug 04 20:42:31.163469 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:31.163434    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:31.166359 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:31.166337    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:31.507136 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:31.507052    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:42:31.509679 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:31.509655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fea94516-a7b5-41cc-84c5-10a37f3d021a-metrics-tls\") pod \"dns-default-f9bch\" (UID: \"fea94516-a7b5-41cc-84c5-10a37f3d021a\") " pod="openshift-dns/dns-default-f9bch"
Aug 04 20:42:31.586114 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:31.586090    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:31.587400 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:31.587380    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6765f794fb-9thrr"
Aug 04 20:42:31.587936 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:31.587914    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kcgl6\""
Aug 04 20:42:31.595229 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:31.595212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f9bch"
Aug 04 20:42:31.787491 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:31.787426    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f9bch"]
Aug 04 20:42:31.790374 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:42:31.790310    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea94516_a7b5_41cc_84c5_10a37f3d021a.slice/crio-49f65653eaf59e534b60a143a0868fb89128f08136cfeede0d79ffdbdb433b52 WatchSource:0}: Error finding container 49f65653eaf59e534b60a143a0868fb89128f08136cfeede0d79ffdbdb433b52: Status 404 returned error can't find the container with id 49f65653eaf59e534b60a143a0868fb89128f08136cfeede0d79ffdbdb433b52
Aug 04 20:42:32.590947 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:32.590913    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh" event={"ID":"3dbdac82-ac1f-49c9-9f64-98c6f63e8076","Type":"ContainerStarted","Data":"48b3e63a092e2cd41b1ab087e4bf42c4959f69142496d9a16e962a19ef04603d"}
Aug 04 20:42:32.592141 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:32.592115    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f9bch" event={"ID":"fea94516-a7b5-41cc-84c5-10a37f3d021a","Type":"ContainerStarted","Data":"49f65653eaf59e534b60a143a0868fb89128f08136cfeede0d79ffdbdb433b52"}
Aug 04 20:42:32.612713 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:32.612673    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-7cwsh" podStartSLOduration=33.082802898 podStartE2EDuration="34.612659294s" totalImagesPullingTime="1.529856396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:41:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:32.611229691 +0000 UTC m=+99.854848974" watchObservedRunningTime="2026-08-04 20:42:32.612659294 +0000 UTC m=+99.856278575"
Aug 04 20:42:33.722454 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:33.722432    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:33.724678 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:33.724659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db41f9ff-f898-4cda-b1b0-2bc2dba136ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f872z\" (UID: \"db41f9ff-f898-4cda-b1b0-2bc2dba136ba\") " pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:33.735652 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:33.735630    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f872z"
Aug 04 20:42:33.880576 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:33.880529    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f872z"]
Aug 04 20:42:33.883045 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:42:33.883014    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb41f9ff_f898_4cda_b1b0_2bc2dba136ba.slice/crio-644838e74a0f9e0566fdcc14b59194ba5ac3201e154deaba88fd727a420c4e7f WatchSource:0}: Error finding container 644838e74a0f9e0566fdcc14b59194ba5ac3201e154deaba88fd727a420c4e7f: Status 404 returned error can't find the container with id 644838e74a0f9e0566fdcc14b59194ba5ac3201e154deaba88fd727a420c4e7f
Aug 04 20:42:34.599549 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:34.599514    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f9bch" event={"ID":"fea94516-a7b5-41cc-84c5-10a37f3d021a","Type":"ContainerStarted","Data":"bb59d4e9b8906884579348e478d6942c0ae4d2c01406a975f178fa44be777a76"}
Aug 04 20:42:34.599705 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:34.599579    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f9bch" event={"ID":"fea94516-a7b5-41cc-84c5-10a37f3d021a","Type":"ContainerStarted","Data":"ef81b0dadb493142b8294232291d86215a1bc1252e70f9a3dfb610eefec0455a"}
Aug 04 20:42:34.599705 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:34.599604    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f9bch"
Aug 04 20:42:34.600925 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:34.600905    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f872z" event={"ID":"db41f9ff-f898-4cda-b1b0-2bc2dba136ba","Type":"ContainerStarted","Data":"b6b694fdd89b6c529cc76d8e27612dab4d2c69fe299b04d720217caf052a1801"}
Aug 04 20:42:34.601009 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:34.600945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f872z" event={"ID":"db41f9ff-f898-4cda-b1b0-2bc2dba136ba","Type":"ContainerStarted","Data":"644838e74a0f9e0566fdcc14b59194ba5ac3201e154deaba88fd727a420c4e7f"}
Aug 04 20:42:34.617190 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:34.617153    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f9bch" podStartSLOduration=65.782935003 podStartE2EDuration="1m7.617140597s" totalImagesPullingTime="1.834205594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:34.616103761 +0000 UTC m=+101.859723044" watchObservedRunningTime="2026-08-04 20:42:34.617140597 +0000 UTC m=+101.860759879"
Aug 04 20:42:35.507006 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:35.506982    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bkp8v"
Aug 04 20:42:35.607733 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:35.607651    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f872z" event={"ID":"db41f9ff-f898-4cda-b1b0-2bc2dba136ba","Type":"ContainerStarted","Data":"507d2efaa6ab2a4882b507b712207e68d674205821ebc6ae90e2cbea075488c4"}
Aug 04 20:42:37.615482 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:37.615451    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f872z" event={"ID":"db41f9ff-f898-4cda-b1b0-2bc2dba136ba","Type":"ContainerStarted","Data":"bb6025b6e8981c00abe564d788409c6e0723eb30a38a57fa8eeea3b582eab2fd"}
Aug 04 20:42:37.636082 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:37.636037    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-f872z" podStartSLOduration=34.064643199 podStartE2EDuration="36.63602363s" totalImagesPullingTime="2.571380431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 20:42:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:37.635899147 +0000 UTC m=+104.879518429" watchObservedRunningTime="2026-08-04 20:42:37.63602363 +0000 UTC m=+104.879642912"
Aug 04 20:42:38.614779 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.614752    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b757f955-dkhmx"]
Aug 04 20:42:38.619535 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.619501    2563 patch_prober.go:28] interesting pod/image-registry-7b757f955-dkhmx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Aug 04 20:42:38.619924 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.619548    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7b757f955-dkhmx" podUID="9d81429a-932e-4050-8b72-4efb1f021150" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 20:42:38.706713 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.706688    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c6f664b95-wkksp"]
Aug 04 20:42:38.709740 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.709726    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.729353 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.729311    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c6f664b95-wkksp"]
Aug 04 20:42:38.759853 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.759830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-ca-trust-extracted\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.759959 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.759855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-installation-pull-secrets\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.759959 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.759887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-bound-sa-token\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.760031 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.759958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-trusted-ca\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.760031 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.760000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-image-registry-private-configuration\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.760031 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.760026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-registry-certificates\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.760165 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.760048    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxgzt\" (UniqueName: \"kubernetes.io/projected/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-kube-api-access-pxgzt\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.760165 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.760152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-registry-tls\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.861421 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.861395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-registry-tls\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.861531 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.861453    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-ca-trust-extracted\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.861531 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.861476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-installation-pull-secrets\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.861531 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.861500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-bound-sa-token\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.861531 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.861527    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-trusted-ca\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.861729 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.861559    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-image-registry-private-configuration\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.861729 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.861594    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-registry-certificates\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.861729 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.861619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxgzt\" (UniqueName: \"kubernetes.io/projected/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-kube-api-access-pxgzt\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.861910 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.861881    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-ca-trust-extracted\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.862418 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.862398    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-registry-certificates\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.862869 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.862841    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-trusted-ca\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.863948 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.863920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-installation-pull-secrets\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.864065 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.863945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-image-registry-private-configuration\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.864188 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.864172    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-registry-tls\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.878345 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.878302    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxgzt\" (UniqueName: \"kubernetes.io/projected/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-kube-api-access-pxgzt\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:38.885139 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:38.885117    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cdeb039d-a981-49dc-bb7c-fe1f126f75b7-bound-sa-token\") pod \"image-registry-7c6f664b95-wkksp\" (UID: \"cdeb039d-a981-49dc-bb7c-fe1f126f75b7\") " pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:39.018134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:39.018114    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:39.152037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:39.151961    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c6f664b95-wkksp"]
Aug 04 20:42:39.154493 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:42:39.154470    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdeb039d_a981_49dc_bb7c_fe1f126f75b7.slice/crio-9347f831751fe999720babf1d9d68603b5932a0e0e02b5113b3c6d9eb1e269b5 WatchSource:0}: Error finding container 9347f831751fe999720babf1d9d68603b5932a0e0e02b5113b3c6d9eb1e269b5: Status 404 returned error can't find the container with id 9347f831751fe999720babf1d9d68603b5932a0e0e02b5113b3c6d9eb1e269b5
Aug 04 20:42:39.623260 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:39.623227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c6f664b95-wkksp" event={"ID":"cdeb039d-a981-49dc-bb7c-fe1f126f75b7","Type":"ContainerStarted","Data":"3935d58e5772d1b72751f1b4a095dcc7c1e92683e3803cc3035d15319be80fb4"}
Aug 04 20:42:39.623625 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:39.623281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c6f664b95-wkksp" event={"ID":"cdeb039d-a981-49dc-bb7c-fe1f126f75b7","Type":"ContainerStarted","Data":"9347f831751fe999720babf1d9d68603b5932a0e0e02b5113b3c6d9eb1e269b5"}
Aug 04 20:42:39.623625 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:39.623357    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:42:39.646146 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:39.646109    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c6f664b95-wkksp" podStartSLOduration=1.646097054 podStartE2EDuration="1.646097054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:39.645369006 +0000 UTC m=+106.888988288" watchObservedRunningTime="2026-08-04 20:42:39.646097054 +0000 UTC m=+106.889716335"
Aug 04 20:42:44.609491 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:44.609459    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f9bch"
Aug 04 20:42:46.694850 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.694821    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zf4cn"]
Aug 04 20:42:46.697668 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.697647    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.701633 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.701610    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 04 20:42:46.701712 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.701610    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 04 20:42:46.702433 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.702410    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 04 20:42:46.702486 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.702446    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 04 20:42:46.702486 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.702476    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cjqnv\""
Aug 04 20:42:46.819538 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.819520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/586b8879-a1db-4d3c-b342-5ea6cef440f8-root\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.819637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.819551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-accelerators-collector-config\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.819637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.819572    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/586b8879-a1db-4d3c-b342-5ea6cef440f8-sys\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.819637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.819593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-textfile\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.819772 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.819656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjrsc\" (UniqueName: \"kubernetes.io/projected/586b8879-a1db-4d3c-b342-5ea6cef440f8-kube-api-access-jjrsc\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.819772 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.819690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/586b8879-a1db-4d3c-b342-5ea6cef440f8-metrics-client-ca\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.819877 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.819776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-wtmp\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.819877 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.819836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-tls\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.819877 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.819866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920439 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/586b8879-a1db-4d3c-b342-5ea6cef440f8-root\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920545 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920453    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-accelerators-collector-config\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920545 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/586b8879-a1db-4d3c-b342-5ea6cef440f8-sys\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920545 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-textfile\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920545 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920535    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/586b8879-a1db-4d3c-b342-5ea6cef440f8-root\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920744 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920541    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/586b8879-a1db-4d3c-b342-5ea6cef440f8-sys\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920744 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjrsc\" (UniqueName: \"kubernetes.io/projected/586b8879-a1db-4d3c-b342-5ea6cef440f8-kube-api-access-jjrsc\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920744 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/586b8879-a1db-4d3c-b342-5ea6cef440f8-metrics-client-ca\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920744 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920652    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-wtmp\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920744 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-tls\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920744 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.920744 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920727    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-textfile\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.921075 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.920802    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-wtmp\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.921075 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:46.920809    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Aug 04 20:42:46.921075 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:42:46.920897    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-tls podName:586b8879-a1db-4d3c-b342-5ea6cef440f8 nodeName:}" failed. No retries permitted until 2026-08-04 20:42:47.420876841 +0000 UTC m=+114.664496115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-tls") pod "node-exporter-zf4cn" (UID: "586b8879-a1db-4d3c-b342-5ea6cef440f8") : secret "node-exporter-tls" not found
Aug 04 20:42:46.921075 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.921023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-accelerators-collector-config\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.921075 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.921040    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/586b8879-a1db-4d3c-b342-5ea6cef440f8-metrics-client-ca\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.923013 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.922993    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:46.932663 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:46.932643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjrsc\" (UniqueName: \"kubernetes.io/projected/586b8879-a1db-4d3c-b342-5ea6cef440f8-kube-api-access-jjrsc\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:47.424504 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:47.424470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-tls\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:47.426689 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:47.426667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/586b8879-a1db-4d3c-b342-5ea6cef440f8-node-exporter-tls\") pod \"node-exporter-zf4cn\" (UID: \"586b8879-a1db-4d3c-b342-5ea6cef440f8\") " pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:47.606908 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:47.606883    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zf4cn"
Aug 04 20:42:47.616225 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:42:47.616207    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod586b8879_a1db_4d3c_b342_5ea6cef440f8.slice/crio-02b5c5d5d18ff92738fcc2c49a0fc1a27310c558a380383ab72049a992a3c76a WatchSource:0}: Error finding container 02b5c5d5d18ff92738fcc2c49a0fc1a27310c558a380383ab72049a992a3c76a: Status 404 returned error can't find the container with id 02b5c5d5d18ff92738fcc2c49a0fc1a27310c558a380383ab72049a992a3c76a
Aug 04 20:42:47.654876 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:47.654849    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zf4cn" event={"ID":"586b8879-a1db-4d3c-b342-5ea6cef440f8","Type":"ContainerStarted","Data":"02b5c5d5d18ff92738fcc2c49a0fc1a27310c558a380383ab72049a992a3c76a"}
Aug 04 20:42:48.619915 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:48.619888    2563 patch_prober.go:28] interesting pod/image-registry-7b757f955-dkhmx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Aug 04 20:42:48.620281 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:48.619932    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7b757f955-dkhmx" podUID="9d81429a-932e-4050-8b72-4efb1f021150" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 20:42:48.660135 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:48.660099    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zf4cn" event={"ID":"586b8879-a1db-4d3c-b342-5ea6cef440f8","Type":"ContainerStarted","Data":"80a990416dd756499245fdd105997f2fb37c274b62b8d0b3a987aae41df89062"}
Aug 04 20:42:49.665043 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:49.665012    2563 generic.go:356] "Generic (PLEG): container finished" podID="586b8879-a1db-4d3c-b342-5ea6cef440f8" containerID="80a990416dd756499245fdd105997f2fb37c274b62b8d0b3a987aae41df89062" exitCode=0
Aug 04 20:42:49.665419 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:49.665056    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zf4cn" event={"ID":"586b8879-a1db-4d3c-b342-5ea6cef440f8","Type":"ContainerDied","Data":"80a990416dd756499245fdd105997f2fb37c274b62b8d0b3a987aae41df89062"}
Aug 04 20:42:50.670331 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:50.670288    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zf4cn" event={"ID":"586b8879-a1db-4d3c-b342-5ea6cef440f8","Type":"ContainerStarted","Data":"ce29ed14d93d8364608c2b6b1d2bc8695ff99482bafcb9780bc46e21d461397e"}
Aug 04 20:42:50.670684 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:50.670400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zf4cn" event={"ID":"586b8879-a1db-4d3c-b342-5ea6cef440f8","Type":"ContainerStarted","Data":"79ff7fcbcb55be9d770c4d448e9b811d8ed5c25e831a381fccd977ab792e860a"}
Aug 04 20:42:50.694036 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:50.693992    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zf4cn" podStartSLOduration=3.643870967 podStartE2EDuration="4.693977451s" totalImagesPullingTime="980.58117ms" totalInitContainerRuntime="69.525314ms" isStatefulPod=true podCreationTimestamp="2026-08-04 20:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:50.6930689 +0000 UTC m=+117.936688183" watchObservedRunningTime="2026-08-04 20:42:50.693977451 +0000 UTC m=+117.937596731"
Aug 04 20:42:51.089368 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.089271    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"]
Aug 04 20:42:51.092344 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.092304    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.094652 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.094621    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Aug 04 20:42:51.096040 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.096025    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5zmnx\""
Aug 04 20:42:51.096196 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.096179    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Aug 04 20:42:51.096276 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.096206    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Aug 04 20:42:51.096276 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.096269    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fmjalle6b8sv1\""
Aug 04 20:42:51.096466 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.096358    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Aug 04 20:42:51.101248 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.101228    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"]
Aug 04 20:42:51.250433 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.250409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/581f4a82-3bde-43ab-a599-ab8198976690-client-ca-bundle\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.250557 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.250438    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/581f4a82-3bde-43ab-a599-ab8198976690-audit-log\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.250557 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.250475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/581f4a82-3bde-43ab-a599-ab8198976690-secret-metrics-server-tls\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.250557 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.250516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581f4a82-3bde-43ab-a599-ab8198976690-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.250648 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.250572    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtgbb\" (UniqueName: \"kubernetes.io/projected/581f4a82-3bde-43ab-a599-ab8198976690-kube-api-access-gtgbb\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.250648 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.250609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/581f4a82-3bde-43ab-a599-ab8198976690-secret-metrics-server-client-certs\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.250648 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.250627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/581f4a82-3bde-43ab-a599-ab8198976690-metrics-server-audit-profiles\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.351375 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.351302    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/581f4a82-3bde-43ab-a599-ab8198976690-client-ca-bundle\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.351375 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.351349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/581f4a82-3bde-43ab-a599-ab8198976690-audit-log\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.351518 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.351390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/581f4a82-3bde-43ab-a599-ab8198976690-secret-metrics-server-tls\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.351518 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.351415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581f4a82-3bde-43ab-a599-ab8198976690-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.351518 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.351447    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtgbb\" (UniqueName: \"kubernetes.io/projected/581f4a82-3bde-43ab-a599-ab8198976690-kube-api-access-gtgbb\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.351656 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.351609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/581f4a82-3bde-43ab-a599-ab8198976690-secret-metrics-server-client-certs\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.351656 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.351643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/581f4a82-3bde-43ab-a599-ab8198976690-metrics-server-audit-profiles\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.351839 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.351816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/581f4a82-3bde-43ab-a599-ab8198976690-audit-log\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.352543 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.352513    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/581f4a82-3bde-43ab-a599-ab8198976690-metrics-server-audit-profiles\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.352790 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.352770    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581f4a82-3bde-43ab-a599-ab8198976690-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.353982 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.353954    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/581f4a82-3bde-43ab-a599-ab8198976690-client-ca-bundle\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.353982 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.353969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/581f4a82-3bde-43ab-a599-ab8198976690-secret-metrics-server-tls\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.354099 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.354039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/581f4a82-3bde-43ab-a599-ab8198976690-secret-metrics-server-client-certs\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.360823 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.360797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtgbb\" (UniqueName: \"kubernetes.io/projected/581f4a82-3bde-43ab-a599-ab8198976690-kube-api-access-gtgbb\") pod \"metrics-server-86cd44d5cf-qzt5j\" (UID: \"581f4a82-3bde-43ab-a599-ab8198976690\") " pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.401207 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.401182    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:42:51.525343 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.525291    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"]
Aug 04 20:42:51.528779 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:42:51.528754    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581f4a82_3bde_43ab_a599_ab8198976690.slice/crio-0b5a6fad7ab0b9569d05080ebf90e6d005c2885609238558136399cdbfea68e5 WatchSource:0}: Error finding container 0b5a6fad7ab0b9569d05080ebf90e6d005c2885609238558136399cdbfea68e5: Status 404 returned error can't find the container with id 0b5a6fad7ab0b9569d05080ebf90e6d005c2885609238558136399cdbfea68e5
Aug 04 20:42:51.674704 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:51.674680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j" event={"ID":"581f4a82-3bde-43ab-a599-ab8198976690","Type":"ContainerStarted","Data":"0b5a6fad7ab0b9569d05080ebf90e6d005c2885609238558136399cdbfea68e5"}
Aug 04 20:42:53.683346 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:53.683297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j" event={"ID":"581f4a82-3bde-43ab-a599-ab8198976690","Type":"ContainerStarted","Data":"779c67756b957ac84ce45a9371013ffa8c2c47cbbf02b3cabb96a2c850a0a2aa"}
Aug 04 20:42:53.701106 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:53.701067    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j" podStartSLOduration=1.488620875 podStartE2EDuration="2.701054292s" totalImagesPullingTime="1.212433417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:42:53.700359605 +0000 UTC m=+120.943978888" watchObservedRunningTime="2026-08-04 20:42:53.701054292 +0000 UTC m=+120.944673572"
Aug 04 20:42:58.619132 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:42:58.619106    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:43:00.632865 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:00.632837    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c6f664b95-wkksp"
Aug 04 20:43:02.936827 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:02.936797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:43:02.939031 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:02.939014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0178830-4009-4007-9829-a2c0aee48925-metrics-certs\") pod \"network-metrics-daemon-8wdv9\" (UID: \"d0178830-4009-4007-9829-a2c0aee48925\") " pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:43:03.058421 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:03.058395    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h8x4x\""
Aug 04 20:43:03.066468 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:03.066450    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8wdv9"
Aug 04 20:43:03.190709 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:03.190646    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8wdv9"]
Aug 04 20:43:03.193522 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:43:03.193498    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0178830_4009_4007_9829_a2c0aee48925.slice/crio-4c9c6bb606f2a4275ccaaea886485f0a8a64781d0340aed8debafd6f0f7bf7b6 WatchSource:0}: Error finding container 4c9c6bb606f2a4275ccaaea886485f0a8a64781d0340aed8debafd6f0f7bf7b6: Status 404 returned error can't find the container with id 4c9c6bb606f2a4275ccaaea886485f0a8a64781d0340aed8debafd6f0f7bf7b6
Aug 04 20:43:03.635879 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:03.635806    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7b757f955-dkhmx" podUID="9d81429a-932e-4050-8b72-4efb1f021150" containerName="registry" containerID="cri-o://4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39" gracePeriod=30
Aug 04 20:43:03.716860 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:03.716829    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8wdv9" event={"ID":"d0178830-4009-4007-9829-a2c0aee48925","Type":"ContainerStarted","Data":"4c9c6bb606f2a4275ccaaea886485f0a8a64781d0340aed8debafd6f0f7bf7b6"}
Aug 04 20:43:04.110024 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.110000    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:43:04.246276 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.246247    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d81429a-932e-4050-8b72-4efb1f021150-ca-trust-extracted\") pod \"9d81429a-932e-4050-8b72-4efb1f021150\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") "
Aug 04 20:43:04.246411 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.246282    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-bound-sa-token\") pod \"9d81429a-932e-4050-8b72-4efb1f021150\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") "
Aug 04 20:43:04.246411 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.246301    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-trusted-ca\") pod \"9d81429a-932e-4050-8b72-4efb1f021150\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") "
Aug 04 20:43:04.246411 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.246378    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87z2q\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-kube-api-access-87z2q\") pod \"9d81429a-932e-4050-8b72-4efb1f021150\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") "
Aug 04 20:43:04.246411 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.246401    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-registry-certificates\") pod \"9d81429a-932e-4050-8b72-4efb1f021150\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") "
Aug 04 20:43:04.246601 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.246431    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls\") pod \"9d81429a-932e-4050-8b72-4efb1f021150\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") "
Aug 04 20:43:04.246601 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.246467    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-installation-pull-secrets\") pod \"9d81429a-932e-4050-8b72-4efb1f021150\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") "
Aug 04 20:43:04.246601 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.246497    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-image-registry-private-configuration\") pod \"9d81429a-932e-4050-8b72-4efb1f021150\" (UID: \"9d81429a-932e-4050-8b72-4efb1f021150\") "
Aug 04 20:43:04.246909 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.246877    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9d81429a-932e-4050-8b72-4efb1f021150" (UID: "9d81429a-932e-4050-8b72-4efb1f021150"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 20:43:04.247229 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.246886    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9d81429a-932e-4050-8b72-4efb1f021150" (UID: "9d81429a-932e-4050-8b72-4efb1f021150"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 20:43:04.248758 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.248731    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9d81429a-932e-4050-8b72-4efb1f021150" (UID: "9d81429a-932e-4050-8b72-4efb1f021150"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:43:04.248859 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.248794    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-kube-api-access-87z2q" (OuterVolumeSpecName: "kube-api-access-87z2q") pod "9d81429a-932e-4050-8b72-4efb1f021150" (UID: "9d81429a-932e-4050-8b72-4efb1f021150"). InnerVolumeSpecName "kube-api-access-87z2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:43:04.249267 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.249247    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9d81429a-932e-4050-8b72-4efb1f021150" (UID: "9d81429a-932e-4050-8b72-4efb1f021150"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 20:43:04.249355 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.249256    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9d81429a-932e-4050-8b72-4efb1f021150" (UID: "9d81429a-932e-4050-8b72-4efb1f021150"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:43:04.249355 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.249293    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9d81429a-932e-4050-8b72-4efb1f021150" (UID: "9d81429a-932e-4050-8b72-4efb1f021150"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 20:43:04.258201 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.258130    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d81429a-932e-4050-8b72-4efb1f021150-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9d81429a-932e-4050-8b72-4efb1f021150" (UID: "9d81429a-932e-4050-8b72-4efb1f021150"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:43:04.347799 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.347772    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87z2q\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-kube-api-access-87z2q\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:43:04.347901 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.347804    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-registry-certificates\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:43:04.347901 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.347819    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-registry-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:43:04.347901 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.347832    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-installation-pull-secrets\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:43:04.347901 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.347848    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d81429a-932e-4050-8b72-4efb1f021150-image-registry-private-configuration\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:43:04.347901 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.347862    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d81429a-932e-4050-8b72-4efb1f021150-ca-trust-extracted\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:43:04.347901 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.347874    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d81429a-932e-4050-8b72-4efb1f021150-bound-sa-token\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:43:04.347901 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.347886    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d81429a-932e-4050-8b72-4efb1f021150-trusted-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:43:04.721534 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.721506    2563 generic.go:356] "Generic (PLEG): container finished" podID="9d81429a-932e-4050-8b72-4efb1f021150" containerID="4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39" exitCode=0
Aug 04 20:43:04.721668 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.721542    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b757f955-dkhmx"
Aug 04 20:43:04.721668 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.721581    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b757f955-dkhmx" event={"ID":"9d81429a-932e-4050-8b72-4efb1f021150","Type":"ContainerDied","Data":"4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39"}
Aug 04 20:43:04.721668 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.721625    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b757f955-dkhmx" event={"ID":"9d81429a-932e-4050-8b72-4efb1f021150","Type":"ContainerDied","Data":"77b09d787c14dd4720dab8d2ebbbe898346164ee1e0cd33e707e0a805917d1b8"}
Aug 04 20:43:04.721668 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.721655    2563 scope.go:117] "RemoveContainer" containerID="4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39"
Aug 04 20:43:04.723649 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.723617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8wdv9" event={"ID":"d0178830-4009-4007-9829-a2c0aee48925","Type":"ContainerStarted","Data":"73622c085b94b6f77c8d7f22e43ce13abdc2cdf073682cf77f825021b873a795"}
Aug 04 20:43:04.723738 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.723666    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8wdv9" event={"ID":"d0178830-4009-4007-9829-a2c0aee48925","Type":"ContainerStarted","Data":"146fc05f51ff7513be32bf0e2554aa3e01ce4aa3d9ef2caf3c9e403f7ce8b969"}
Aug 04 20:43:04.729570 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.729554    2563 scope.go:117] "RemoveContainer" containerID="4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39"
Aug 04 20:43:04.729823 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:43:04.729805    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39\": container with ID starting with 4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39 not found: ID does not exist" containerID="4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39"
Aug 04 20:43:04.729872 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.729830    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39"} err="failed to get container status \"4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39\": rpc error: code = NotFound desc = could not find container \"4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39\": container with ID starting with 4af63f617f45a45241576fbf13e70a914c088d6c4c2f6b0638783ff789cd6c39 not found: ID does not exist"
Aug 04 20:43:04.742147 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.742111    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8wdv9" podStartSLOduration=130.674437587 podStartE2EDuration="2m11.742098394s" totalImagesPullingTime="1.067660807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:43:04.740599231 +0000 UTC m=+131.984218513" watchObservedRunningTime="2026-08-04 20:43:04.742098394 +0000 UTC m=+131.985717710"
Aug 04 20:43:04.755598 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.755578    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b757f955-dkhmx"]
Aug 04 20:43:04.758918 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:04.758901    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7b757f955-dkhmx"]
Aug 04 20:43:05.241826 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:05.241801    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d81429a-932e-4050-8b72-4efb1f021150" path="/var/lib/kubelet/pods/9d81429a-932e-4050-8b72-4efb1f021150/volumes"
Aug 04 20:43:11.401902 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:11.401873    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:43:11.402333 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:11.401940    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:43:31.406384 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:31.406357    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:43:31.409958 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:43:31.409937    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-86cd44d5cf-qzt5j"
Aug 04 20:45:53.103749 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:45:53.103722    2563 kubelet.go:1648] "Image garbage collection succeeded"
Aug 04 20:46:39.379078 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.379047    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"]
Aug 04 20:46:39.379494 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.379256    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d81429a-932e-4050-8b72-4efb1f021150" containerName="registry"
Aug 04 20:46:39.379494 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.379266    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d81429a-932e-4050-8b72-4efb1f021150" containerName="registry"
Aug 04 20:46:39.379494 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.379311    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d81429a-932e-4050-8b72-4efb1f021150" containerName="registry"
Aug 04 20:46:39.382004 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.381983    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:39.384402 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.384371    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Aug 04 20:46:39.385283 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.385270    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7dxrk\""
Aug 04 20:46:39.385402 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.385350    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Aug 04 20:46:39.390952 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.390933    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"]
Aug 04 20:46:39.466524 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.466500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:39.466616 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.466536    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:39.466616 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.466564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt6hd\" (UniqueName: \"kubernetes.io/projected/0c9b4b2a-905c-4489-8468-b121ec45a0d1-kube-api-access-jt6hd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:39.566943 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.566918    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt6hd\" (UniqueName: \"kubernetes.io/projected/0c9b4b2a-905c-4489-8468-b121ec45a0d1-kube-api-access-jt6hd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:39.567034 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.566957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:39.567034 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.566984    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:39.567249 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.567236    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:39.567289 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.567265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:39.575126 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.575109    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt6hd\" (UniqueName: \"kubernetes.io/projected/0c9b4b2a-905c-4489-8468-b121ec45a0d1-kube-api-access-jt6hd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:39.691620 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.691564    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:39.814231 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.814210    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"]
Aug 04 20:46:39.816726 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:46:39.816699    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9b4b2a_905c_4489_8468_b121ec45a0d1.slice/crio-4208c9d9863d391ff9d984182a8fbaf8ac0a67d9b037d4ba3a41fd877f7d0485 WatchSource:0}: Error finding container 4208c9d9863d391ff9d984182a8fbaf8ac0a67d9b037d4ba3a41fd877f7d0485: Status 404 returned error can't find the container with id 4208c9d9863d391ff9d984182a8fbaf8ac0a67d9b037d4ba3a41fd877f7d0485
Aug 04 20:46:39.818424 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:39.818409    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 20:46:40.390026 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:40.389991    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv" event={"ID":"0c9b4b2a-905c-4489-8468-b121ec45a0d1","Type":"ContainerStarted","Data":"4208c9d9863d391ff9d984182a8fbaf8ac0a67d9b037d4ba3a41fd877f7d0485"}
Aug 04 20:46:49.424191 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:49.424157    2563 generic.go:356] "Generic (PLEG): container finished" podID="0c9b4b2a-905c-4489-8468-b121ec45a0d1" containerID="8eeb334362a798a3d9e56658f4f4c7fbd1d0742c0cf6f4fd475bbe4780a1002b" exitCode=0
Aug 04 20:46:49.424575 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:49.424199    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv" event={"ID":"0c9b4b2a-905c-4489-8468-b121ec45a0d1","Type":"ContainerDied","Data":"8eeb334362a798a3d9e56658f4f4c7fbd1d0742c0cf6f4fd475bbe4780a1002b"}
Aug 04 20:46:52.436113 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:52.436080    2563 generic.go:356] "Generic (PLEG): container finished" podID="0c9b4b2a-905c-4489-8468-b121ec45a0d1" containerID="a4fbd8ee9f1ab63e362fbf1c2872c2e6dfed22a7ad35b7a98917474454a13857" exitCode=0
Aug 04 20:46:52.436452 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:52.436164    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv" event={"ID":"0c9b4b2a-905c-4489-8468-b121ec45a0d1","Type":"ContainerDied","Data":"a4fbd8ee9f1ab63e362fbf1c2872c2e6dfed22a7ad35b7a98917474454a13857"}
Aug 04 20:46:58.459565 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:58.459531    2563 generic.go:356] "Generic (PLEG): container finished" podID="0c9b4b2a-905c-4489-8468-b121ec45a0d1" containerID="f0c9bc55ca12b3d9360ffca5e93f8a2ebbb747b7b23776e2437c85ee927689b7" exitCode=0
Aug 04 20:46:58.459565 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:58.459566    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv" event={"ID":"0c9b4b2a-905c-4489-8468-b121ec45a0d1","Type":"ContainerDied","Data":"f0c9bc55ca12b3d9360ffca5e93f8a2ebbb747b7b23776e2437c85ee927689b7"}
Aug 04 20:46:59.579348 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:59.579299    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:46:59.715250 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:59.715197    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt6hd\" (UniqueName: \"kubernetes.io/projected/0c9b4b2a-905c-4489-8468-b121ec45a0d1-kube-api-access-jt6hd\") pod \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") "
Aug 04 20:46:59.715384 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:59.715267    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-bundle\") pod \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") "
Aug 04 20:46:59.715384 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:59.715366    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-util\") pod \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\" (UID: \"0c9b4b2a-905c-4489-8468-b121ec45a0d1\") "
Aug 04 20:46:59.715912 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:59.715893    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-bundle" (OuterVolumeSpecName: "bundle") pod "0c9b4b2a-905c-4489-8468-b121ec45a0d1" (UID: "0c9b4b2a-905c-4489-8468-b121ec45a0d1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:46:59.717446 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:59.717424    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9b4b2a-905c-4489-8468-b121ec45a0d1-kube-api-access-jt6hd" (OuterVolumeSpecName: "kube-api-access-jt6hd") pod "0c9b4b2a-905c-4489-8468-b121ec45a0d1" (UID: "0c9b4b2a-905c-4489-8468-b121ec45a0d1"). InnerVolumeSpecName "kube-api-access-jt6hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:46:59.719524 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:59.719504    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-util" (OuterVolumeSpecName: "util") pod "0c9b4b2a-905c-4489-8468-b121ec45a0d1" (UID: "0c9b4b2a-905c-4489-8468-b121ec45a0d1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:46:59.816255 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:59.816236    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-bundle\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:46:59.816255 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:59.816253    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c9b4b2a-905c-4489-8468-b121ec45a0d1-util\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:46:59.816392 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:46:59.816261    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt6hd\" (UniqueName: \"kubernetes.io/projected/0c9b4b2a-905c-4489-8468-b121ec45a0d1-kube-api-access-jt6hd\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:47:00.468657 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:00.468637    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv"
Aug 04 20:47:00.468657 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:00.468650    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lxkhv" event={"ID":"0c9b4b2a-905c-4489-8468-b121ec45a0d1","Type":"ContainerDied","Data":"4208c9d9863d391ff9d984182a8fbaf8ac0a67d9b037d4ba3a41fd877f7d0485"}
Aug 04 20:47:00.468817 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:00.468685    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4208c9d9863d391ff9d984182a8fbaf8ac0a67d9b037d4ba3a41fd877f7d0485"
Aug 04 20:47:07.449119 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.449079    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd"]
Aug 04 20:47:07.449733 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.449420    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c9b4b2a-905c-4489-8468-b121ec45a0d1" containerName="extract"
Aug 04 20:47:07.449733 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.449438    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c9b4b2a-905c-4489-8468-b121ec45a0d1" containerName="extract"
Aug 04 20:47:07.449733 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.449454    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c9b4b2a-905c-4489-8468-b121ec45a0d1" containerName="pull"
Aug 04 20:47:07.449733 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.449461    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c9b4b2a-905c-4489-8468-b121ec45a0d1" containerName="pull"
Aug 04 20:47:07.449733 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.449472    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c9b4b2a-905c-4489-8468-b121ec45a0d1" containerName="util"
Aug 04 20:47:07.449733 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.449481    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c9b4b2a-905c-4489-8468-b121ec45a0d1" containerName="util"
Aug 04 20:47:07.449733 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.449542    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c9b4b2a-905c-4489-8468-b121ec45a0d1" containerName="extract"
Aug 04 20:47:07.454034 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.454014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd"
Aug 04 20:47:07.456911 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.456888    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Aug 04 20:47:07.457027 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.456954    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qf98t\""
Aug 04 20:47:07.457287 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.457273    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Aug 04 20:47:07.466221 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.466202    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd"]
Aug 04 20:47:07.573578 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.573552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94c3fc8e-4990-49da-ada0-7aa1397316bd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-49cjd\" (UID: \"94c3fc8e-4990-49da-ada0-7aa1397316bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd"
Aug 04 20:47:07.573712 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.573584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t77x2\" (UniqueName: \"kubernetes.io/projected/94c3fc8e-4990-49da-ada0-7aa1397316bd-kube-api-access-t77x2\") pod \"cert-manager-operator-controller-manager-55c444bf86-49cjd\" (UID: \"94c3fc8e-4990-49da-ada0-7aa1397316bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd"
Aug 04 20:47:07.674241 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.674210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94c3fc8e-4990-49da-ada0-7aa1397316bd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-49cjd\" (UID: \"94c3fc8e-4990-49da-ada0-7aa1397316bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd"
Aug 04 20:47:07.674366 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.674248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t77x2\" (UniqueName: \"kubernetes.io/projected/94c3fc8e-4990-49da-ada0-7aa1397316bd-kube-api-access-t77x2\") pod \"cert-manager-operator-controller-manager-55c444bf86-49cjd\" (UID: \"94c3fc8e-4990-49da-ada0-7aa1397316bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd"
Aug 04 20:47:07.674611 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.674591    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94c3fc8e-4990-49da-ada0-7aa1397316bd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-49cjd\" (UID: \"94c3fc8e-4990-49da-ada0-7aa1397316bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd"
Aug 04 20:47:07.682717 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.682699    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t77x2\" (UniqueName: \"kubernetes.io/projected/94c3fc8e-4990-49da-ada0-7aa1397316bd-kube-api-access-t77x2\") pod \"cert-manager-operator-controller-manager-55c444bf86-49cjd\" (UID: \"94c3fc8e-4990-49da-ada0-7aa1397316bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd"
Aug 04 20:47:07.762202 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.762146    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd"
Aug 04 20:47:07.887570 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:07.887551    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd"]
Aug 04 20:47:07.890382 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:47:07.890353    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94c3fc8e_4990_49da_ada0_7aa1397316bd.slice/crio-9d123c711d92618c757b2c5df55dcd3f370b02e858f08093b1d778c8be88564b WatchSource:0}: Error finding container 9d123c711d92618c757b2c5df55dcd3f370b02e858f08093b1d778c8be88564b: Status 404 returned error can't find the container with id 9d123c711d92618c757b2c5df55dcd3f370b02e858f08093b1d778c8be88564b
Aug 04 20:47:08.500494 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:08.500461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd" event={"ID":"94c3fc8e-4990-49da-ada0-7aa1397316bd","Type":"ContainerStarted","Data":"9d123c711d92618c757b2c5df55dcd3f370b02e858f08093b1d778c8be88564b"}
Aug 04 20:47:10.508652 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:10.508618    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd" event={"ID":"94c3fc8e-4990-49da-ada0-7aa1397316bd","Type":"ContainerStarted","Data":"5016552920c03ce36a064c48b87ba9ac46dbe4f97ce36a574222a85aadd5e538"}
Aug 04 20:47:10.530759 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:10.530718    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-49cjd" podStartSLOduration=1.6969613799999999 podStartE2EDuration="3.53070567s" totalImagesPullingTime="1.83374429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:47:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:47:10.529464937 +0000 UTC m=+377.773084210" watchObservedRunningTime="2026-08-04 20:47:10.53070567 +0000 UTC m=+377.774324951"
Aug 04 20:47:11.736268 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:11.736239    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"]
Aug 04 20:47:11.738386 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:11.738371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:11.740685 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:11.740666    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Aug 04 20:47:11.740799 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:11.740762    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Aug 04 20:47:11.741826 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:11.741805    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7dxrk\""
Aug 04 20:47:11.750834 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:11.750812    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"]
Aug 04 20:47:11.901446 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:11.901419    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhctj\" (UniqueName: \"kubernetes.io/projected/d4d1dda3-764d-4050-8fb2-94c347c0df91-kube-api-access-zhctj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:11.901610 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:11.901462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:11.901610 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:11.901498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:12.002742 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.002692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:12.002835 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.002735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhctj\" (UniqueName: \"kubernetes.io/projected/d4d1dda3-764d-4050-8fb2-94c347c0df91-kube-api-access-zhctj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:12.002835 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.002786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:12.003094 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.003059    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:12.003155 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.003106    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:12.011334 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.011303    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhctj\" (UniqueName: \"kubernetes.io/projected/d4d1dda3-764d-4050-8fb2-94c347c0df91-kube-api-access-zhctj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:12.046410 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.046389    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:12.174464 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.174433    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"]
Aug 04 20:47:12.177085 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:47:12.177049    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d1dda3_764d_4050_8fb2_94c347c0df91.slice/crio-7aec9718902489995e924f542a28864ffe347c2adcb83c4d81f5217ad3574d34 WatchSource:0}: Error finding container 7aec9718902489995e924f542a28864ffe347c2adcb83c4d81f5217ad3574d34: Status 404 returned error can't find the container with id 7aec9718902489995e924f542a28864ffe347c2adcb83c4d81f5217ad3574d34
Aug 04 20:47:12.517481 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.517453    2563 generic.go:356] "Generic (PLEG): container finished" podID="d4d1dda3-764d-4050-8fb2-94c347c0df91" containerID="44de222eb7c61dddfc1a0a38a7c750ee1c9e94c6d7888359f63ee4598c27a850" exitCode=0
Aug 04 20:47:12.517618 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.517492    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm" event={"ID":"d4d1dda3-764d-4050-8fb2-94c347c0df91","Type":"ContainerDied","Data":"44de222eb7c61dddfc1a0a38a7c750ee1c9e94c6d7888359f63ee4598c27a850"}
Aug 04 20:47:12.517618 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.517529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm" event={"ID":"d4d1dda3-764d-4050-8fb2-94c347c0df91","Type":"ContainerStarted","Data":"7aec9718902489995e924f542a28864ffe347c2adcb83c4d81f5217ad3574d34"}
Aug 04 20:47:12.928226 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.928196    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc"]
Aug 04 20:47:12.930071 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.930056    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc"
Aug 04 20:47:12.932666 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.932644    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mlphv\""
Aug 04 20:47:12.932802 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.932645    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Aug 04 20:47:12.932802 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.932691    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Aug 04 20:47:12.942196 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:12.942178    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc"]
Aug 04 20:47:13.111248 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:13.111226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cm55\" (UniqueName: \"kubernetes.io/projected/3d5a3ea9-d95c-4c85-b585-909d3e678b83-kube-api-access-7cm55\") pod \"cert-manager-cainjector-559cdd8c87-tqhkc\" (UID: \"3d5a3ea9-d95c-4c85-b585-909d3e678b83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc"
Aug 04 20:47:13.111375 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:13.111259    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d5a3ea9-d95c-4c85-b585-909d3e678b83-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tqhkc\" (UID: \"3d5a3ea9-d95c-4c85-b585-909d3e678b83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc"
Aug 04 20:47:13.212180 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:13.212107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cm55\" (UniqueName: \"kubernetes.io/projected/3d5a3ea9-d95c-4c85-b585-909d3e678b83-kube-api-access-7cm55\") pod \"cert-manager-cainjector-559cdd8c87-tqhkc\" (UID: \"3d5a3ea9-d95c-4c85-b585-909d3e678b83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc"
Aug 04 20:47:13.212180 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:13.212160    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d5a3ea9-d95c-4c85-b585-909d3e678b83-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tqhkc\" (UID: \"3d5a3ea9-d95c-4c85-b585-909d3e678b83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc"
Aug 04 20:47:13.221260 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:13.221238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d5a3ea9-d95c-4c85-b585-909d3e678b83-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tqhkc\" (UID: \"3d5a3ea9-d95c-4c85-b585-909d3e678b83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc"
Aug 04 20:47:13.221466 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:13.221446    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cm55\" (UniqueName: \"kubernetes.io/projected/3d5a3ea9-d95c-4c85-b585-909d3e678b83-kube-api-access-7cm55\") pod \"cert-manager-cainjector-559cdd8c87-tqhkc\" (UID: \"3d5a3ea9-d95c-4c85-b585-909d3e678b83\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc"
Aug 04 20:47:13.238690 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:13.238668    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc"
Aug 04 20:47:13.369912 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:13.369787    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc"]
Aug 04 20:47:13.372166 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:47:13.372136    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d5a3ea9_d95c_4c85_b585_909d3e678b83.slice/crio-421740bbde44757732997ba67586ac00e5787db903c3d8faece179804fa1338d WatchSource:0}: Error finding container 421740bbde44757732997ba67586ac00e5787db903c3d8faece179804fa1338d: Status 404 returned error can't find the container with id 421740bbde44757732997ba67586ac00e5787db903c3d8faece179804fa1338d
Aug 04 20:47:13.524239 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:13.524173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc" event={"ID":"3d5a3ea9-d95c-4c85-b585-909d3e678b83","Type":"ContainerStarted","Data":"421740bbde44757732997ba67586ac00e5787db903c3d8faece179804fa1338d"}
Aug 04 20:47:14.531182 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:14.531153    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm" event={"ID":"d4d1dda3-764d-4050-8fb2-94c347c0df91","Type":"ContainerStarted","Data":"d2210dfffc764bc1dd13f0da29f72deaebae94b03bc41256474199fb4113477a"}
Aug 04 20:47:15.537450 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:15.537418    2563 generic.go:356] "Generic (PLEG): container finished" podID="d4d1dda3-764d-4050-8fb2-94c347c0df91" containerID="d2210dfffc764bc1dd13f0da29f72deaebae94b03bc41256474199fb4113477a" exitCode=0
Aug 04 20:47:15.537877 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:15.537509    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm" event={"ID":"d4d1dda3-764d-4050-8fb2-94c347c0df91","Type":"ContainerDied","Data":"d2210dfffc764bc1dd13f0da29f72deaebae94b03bc41256474199fb4113477a"}
Aug 04 20:47:16.543372 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:16.543341    2563 generic.go:356] "Generic (PLEG): container finished" podID="d4d1dda3-764d-4050-8fb2-94c347c0df91" containerID="bc59cbfa750ed0b1f2694b166e58659d12bca88ef1692c11a8d68dcacbc10c2d" exitCode=0
Aug 04 20:47:16.543672 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:16.543423    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm" event={"ID":"d4d1dda3-764d-4050-8fb2-94c347c0df91","Type":"ContainerDied","Data":"bc59cbfa750ed0b1f2694b166e58659d12bca88ef1692c11a8d68dcacbc10c2d"}
Aug 04 20:47:17.548935 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.548901    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc" event={"ID":"3d5a3ea9-d95c-4c85-b585-909d3e678b83","Type":"ContainerStarted","Data":"fe3a84dfabbe34631748c254b9f668c2cc948062acf31a345eae127095ca53a4"}
Aug 04 20:47:17.567304 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.567243    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tqhkc" podStartSLOduration=2.447850896 podStartE2EDuration="5.567226541s" totalImagesPullingTime="3.119375645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:47:17.56592182 +0000 UTC m=+384.809541101" watchObservedRunningTime="2026-08-04 20:47:17.567226541 +0000 UTC m=+384.810845822"
Aug 04 20:47:17.672079 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.672057    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:17.848352 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.848284    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhctj\" (UniqueName: \"kubernetes.io/projected/d4d1dda3-764d-4050-8fb2-94c347c0df91-kube-api-access-zhctj\") pod \"d4d1dda3-764d-4050-8fb2-94c347c0df91\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") "
Aug 04 20:47:17.848352 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.848347    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-bundle\") pod \"d4d1dda3-764d-4050-8fb2-94c347c0df91\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") "
Aug 04 20:47:17.848498 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.848409    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-util\") pod \"d4d1dda3-764d-4050-8fb2-94c347c0df91\" (UID: \"d4d1dda3-764d-4050-8fb2-94c347c0df91\") "
Aug 04 20:47:17.848755 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.848736    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-bundle" (OuterVolumeSpecName: "bundle") pod "d4d1dda3-764d-4050-8fb2-94c347c0df91" (UID: "d4d1dda3-764d-4050-8fb2-94c347c0df91"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:47:17.850421 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.850402    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4d1dda3-764d-4050-8fb2-94c347c0df91-kube-api-access-zhctj" (OuterVolumeSpecName: "kube-api-access-zhctj") pod "d4d1dda3-764d-4050-8fb2-94c347c0df91" (UID: "d4d1dda3-764d-4050-8fb2-94c347c0df91"). InnerVolumeSpecName "kube-api-access-zhctj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:47:17.854950 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.854927    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-util" (OuterVolumeSpecName: "util") pod "d4d1dda3-764d-4050-8fb2-94c347c0df91" (UID: "d4d1dda3-764d-4050-8fb2-94c347c0df91"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:47:17.949478 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.949457    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-util\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:47:17.949478 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.949475    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhctj\" (UniqueName: \"kubernetes.io/projected/d4d1dda3-764d-4050-8fb2-94c347c0df91-kube-api-access-zhctj\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:47:17.949599 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:17.949484    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4d1dda3-764d-4050-8fb2-94c347c0df91-bundle\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:47:18.554789 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:18.554764    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm"
Aug 04 20:47:18.555216 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:18.554795    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn2tpm" event={"ID":"d4d1dda3-764d-4050-8fb2-94c347c0df91","Type":"ContainerDied","Data":"7aec9718902489995e924f542a28864ffe347c2adcb83c4d81f5217ad3574d34"}
Aug 04 20:47:18.555216 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:18.554851    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7aec9718902489995e924f542a28864ffe347c2adcb83c4d81f5217ad3574d34"
Aug 04 20:47:24.323863 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.323831    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj"]
Aug 04 20:47:24.324219 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.324040    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4d1dda3-764d-4050-8fb2-94c347c0df91" containerName="pull"
Aug 04 20:47:24.324219 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.324050    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4d1dda3-764d-4050-8fb2-94c347c0df91" containerName="pull"
Aug 04 20:47:24.324219 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.324068    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4d1dda3-764d-4050-8fb2-94c347c0df91" containerName="util"
Aug 04 20:47:24.324219 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.324073    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4d1dda3-764d-4050-8fb2-94c347c0df91" containerName="util"
Aug 04 20:47:24.324219 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.324079    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4d1dda3-764d-4050-8fb2-94c347c0df91" containerName="extract"
Aug 04 20:47:24.324219 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.324084    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4d1dda3-764d-4050-8fb2-94c347c0df91" containerName="extract"
Aug 04 20:47:24.324219 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.324121    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4d1dda3-764d-4050-8fb2-94c347c0df91" containerName="extract"
Aug 04 20:47:24.327518 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.327503    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj"
Aug 04 20:47:24.329887 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.329862    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Aug 04 20:47:24.329887 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.329872    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Aug 04 20:47:24.330037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.329953    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-wd587\""
Aug 04 20:47:24.337061 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.337040    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj"]
Aug 04 20:47:24.392772 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.392744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce7c5b5f-e412-4e4b-bf21-c4007b9395d1-tmp\") pod \"openshift-lws-operator-fd8ccff4c-l22tj\" (UID: \"ce7c5b5f-e412-4e4b-bf21-c4007b9395d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj"
Aug 04 20:47:24.392859 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.392778    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8492d\" (UniqueName: \"kubernetes.io/projected/ce7c5b5f-e412-4e4b-bf21-c4007b9395d1-kube-api-access-8492d\") pod \"openshift-lws-operator-fd8ccff4c-l22tj\" (UID: \"ce7c5b5f-e412-4e4b-bf21-c4007b9395d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj"
Aug 04 20:47:24.494428 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.494399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce7c5b5f-e412-4e4b-bf21-c4007b9395d1-tmp\") pod \"openshift-lws-operator-fd8ccff4c-l22tj\" (UID: \"ce7c5b5f-e412-4e4b-bf21-c4007b9395d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj"
Aug 04 20:47:24.494521 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.494460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8492d\" (UniqueName: \"kubernetes.io/projected/ce7c5b5f-e412-4e4b-bf21-c4007b9395d1-kube-api-access-8492d\") pod \"openshift-lws-operator-fd8ccff4c-l22tj\" (UID: \"ce7c5b5f-e412-4e4b-bf21-c4007b9395d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj"
Aug 04 20:47:24.495146 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.495121    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce7c5b5f-e412-4e4b-bf21-c4007b9395d1-tmp\") pod \"openshift-lws-operator-fd8ccff4c-l22tj\" (UID: \"ce7c5b5f-e412-4e4b-bf21-c4007b9395d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj"
Aug 04 20:47:24.503511 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.503485    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8492d\" (UniqueName: \"kubernetes.io/projected/ce7c5b5f-e412-4e4b-bf21-c4007b9395d1-kube-api-access-8492d\") pod \"openshift-lws-operator-fd8ccff4c-l22tj\" (UID: \"ce7c5b5f-e412-4e4b-bf21-c4007b9395d1\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj"
Aug 04 20:47:24.635763 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.635744    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj"
Aug 04 20:47:24.761731 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:24.761709    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj"]
Aug 04 20:47:24.764218 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:47:24.764189    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce7c5b5f_e412_4e4b_bf21_c4007b9395d1.slice/crio-b41a98b938161c4152495debc42791bcad2ecd880140aa1b9d0c883c7f82c6f6 WatchSource:0}: Error finding container b41a98b938161c4152495debc42791bcad2ecd880140aa1b9d0c883c7f82c6f6: Status 404 returned error can't find the container with id b41a98b938161c4152495debc42791bcad2ecd880140aa1b9d0c883c7f82c6f6
Aug 04 20:47:25.581487 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:25.581444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj" event={"ID":"ce7c5b5f-e412-4e4b-bf21-c4007b9395d1","Type":"ContainerStarted","Data":"b41a98b938161c4152495debc42791bcad2ecd880140aa1b9d0c883c7f82c6f6"}
Aug 04 20:47:27.592179 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:27.592104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj" event={"ID":"ce7c5b5f-e412-4e4b-bf21-c4007b9395d1","Type":"ContainerStarted","Data":"76b01d23297dc8d71e2cf7c91bd0366c77c60559ee4b919bb331401b758cbfc0"}
Aug 04 20:47:27.611097 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:27.611057    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l22tj" podStartSLOduration=1.113816098 podStartE2EDuration="3.611042336s" totalImagesPullingTime="2.497226238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:47:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:47:27.610214348 +0000 UTC m=+394.853833627" watchObservedRunningTime="2026-08-04 20:47:27.611042336 +0000 UTC m=+394.854661617"
Aug 04 20:47:30.589492 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.589462    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"]
Aug 04 20:47:30.610202 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.610165    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"]
Aug 04 20:47:30.610357 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.610258    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:30.613289 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.613266    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Aug 04 20:47:30.613411 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.613294    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Aug 04 20:47:30.613411 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.613268    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7dxrk\""
Aug 04 20:47:30.640414 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.640392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:30.640529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.640434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:30.640529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.640476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4vwd\" (UniqueName: \"kubernetes.io/projected/882d7c02-ce9e-4504-81ee-2a5c39673d0e-kube-api-access-k4vwd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:30.741236 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.741212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4vwd\" (UniqueName: \"kubernetes.io/projected/882d7c02-ce9e-4504-81ee-2a5c39673d0e-kube-api-access-k4vwd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:30.741362 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.741259    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:30.741362 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.741279    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:30.741587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.741572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:30.741647 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.741606    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:30.750049 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.750032    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4vwd\" (UniqueName: \"kubernetes.io/projected/882d7c02-ce9e-4504-81ee-2a5c39673d0e-kube-api-access-k4vwd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:30.919241 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:30.919220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:31.047115 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:31.047093    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"]
Aug 04 20:47:31.048913 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:47:31.048888    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882d7c02_ce9e_4504_81ee_2a5c39673d0e.slice/crio-01f085799f23277a547e0a6ed32eeb5cecee6f67b332981da5beb8f6093deb1d WatchSource:0}: Error finding container 01f085799f23277a547e0a6ed32eeb5cecee6f67b332981da5beb8f6093deb1d: Status 404 returned error can't find the container with id 01f085799f23277a547e0a6ed32eeb5cecee6f67b332981da5beb8f6093deb1d
Aug 04 20:47:31.608988 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:31.608959    2563 generic.go:356] "Generic (PLEG): container finished" podID="882d7c02-ce9e-4504-81ee-2a5c39673d0e" containerID="4417aa9d082f833f3630c24c33920eab6c6eefb49acebfab5484b78b27998956" exitCode=0
Aug 04 20:47:31.609301 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:31.608999    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb" event={"ID":"882d7c02-ce9e-4504-81ee-2a5c39673d0e","Type":"ContainerDied","Data":"4417aa9d082f833f3630c24c33920eab6c6eefb49acebfab5484b78b27998956"}
Aug 04 20:47:31.609301 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:31.609040    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb" event={"ID":"882d7c02-ce9e-4504-81ee-2a5c39673d0e","Type":"ContainerStarted","Data":"01f085799f23277a547e0a6ed32eeb5cecee6f67b332981da5beb8f6093deb1d"}
Aug 04 20:47:32.613858 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:32.613800    2563 generic.go:356] "Generic (PLEG): container finished" podID="882d7c02-ce9e-4504-81ee-2a5c39673d0e" containerID="9f2e90698e3252f1f4624616c0e1af07c9beebad5d7e5847c5fc2c147112ecb7" exitCode=0
Aug 04 20:47:32.614175 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:32.613875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb" event={"ID":"882d7c02-ce9e-4504-81ee-2a5c39673d0e","Type":"ContainerDied","Data":"9f2e90698e3252f1f4624616c0e1af07c9beebad5d7e5847c5fc2c147112ecb7"}
Aug 04 20:47:33.619532 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:33.619494    2563 generic.go:356] "Generic (PLEG): container finished" podID="882d7c02-ce9e-4504-81ee-2a5c39673d0e" containerID="bb4a97ce75e773b5144ef9de3ca5643fa05a7c7031b310a74c538a08ec45dd30" exitCode=0
Aug 04 20:47:33.619532 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:33.619531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb" event={"ID":"882d7c02-ce9e-4504-81ee-2a5c39673d0e","Type":"ContainerDied","Data":"bb4a97ce75e773b5144ef9de3ca5643fa05a7c7031b310a74c538a08ec45dd30"}
Aug 04 20:47:34.746511 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:34.746491    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:34.766243 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:34.766223    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-bundle\") pod \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") "
Aug 04 20:47:34.766367 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:34.766255    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4vwd\" (UniqueName: \"kubernetes.io/projected/882d7c02-ce9e-4504-81ee-2a5c39673d0e-kube-api-access-k4vwd\") pod \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") "
Aug 04 20:47:34.766367 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:34.766303    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-util\") pod \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\" (UID: \"882d7c02-ce9e-4504-81ee-2a5c39673d0e\") "
Aug 04 20:47:34.766986 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:34.766961    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-bundle" (OuterVolumeSpecName: "bundle") pod "882d7c02-ce9e-4504-81ee-2a5c39673d0e" (UID: "882d7c02-ce9e-4504-81ee-2a5c39673d0e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:47:34.768530 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:34.768482    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/882d7c02-ce9e-4504-81ee-2a5c39673d0e-kube-api-access-k4vwd" (OuterVolumeSpecName: "kube-api-access-k4vwd") pod "882d7c02-ce9e-4504-81ee-2a5c39673d0e" (UID: "882d7c02-ce9e-4504-81ee-2a5c39673d0e"). InnerVolumeSpecName "kube-api-access-k4vwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:47:34.772168 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:34.772146    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-util" (OuterVolumeSpecName: "util") pod "882d7c02-ce9e-4504-81ee-2a5c39673d0e" (UID: "882d7c02-ce9e-4504-81ee-2a5c39673d0e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:47:34.867284 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:34.867265    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-bundle\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:47:34.867284 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:34.867284    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4vwd\" (UniqueName: \"kubernetes.io/projected/882d7c02-ce9e-4504-81ee-2a5c39673d0e-kube-api-access-k4vwd\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:47:34.867454 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:34.867293    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/882d7c02-ce9e-4504-81ee-2a5c39673d0e-util\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:47:35.629298 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:35.629278    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb"
Aug 04 20:47:35.629483 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:35.629275    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmqtjb" event={"ID":"882d7c02-ce9e-4504-81ee-2a5c39673d0e","Type":"ContainerDied","Data":"01f085799f23277a547e0a6ed32eeb5cecee6f67b332981da5beb8f6093deb1d"}
Aug 04 20:47:35.629483 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:35.629394    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01f085799f23277a547e0a6ed32eeb5cecee6f67b332981da5beb8f6093deb1d"
Aug 04 20:47:47.595113 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.595080    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"]
Aug 04 20:47:47.595529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.595344    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="882d7c02-ce9e-4504-81ee-2a5c39673d0e" containerName="pull"
Aug 04 20:47:47.595529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.595355    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="882d7c02-ce9e-4504-81ee-2a5c39673d0e" containerName="pull"
Aug 04 20:47:47.595529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.595363    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="882d7c02-ce9e-4504-81ee-2a5c39673d0e" containerName="extract"
Aug 04 20:47:47.595529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.595369    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="882d7c02-ce9e-4504-81ee-2a5c39673d0e" containerName="extract"
Aug 04 20:47:47.595529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.595376    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="882d7c02-ce9e-4504-81ee-2a5c39673d0e" containerName="util"
Aug 04 20:47:47.595529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.595380    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="882d7c02-ce9e-4504-81ee-2a5c39673d0e" containerName="util"
Aug 04 20:47:47.595529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.595435    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="882d7c02-ce9e-4504-81ee-2a5c39673d0e" containerName="extract"
Aug 04 20:47:47.597287 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.597273    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.600695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.600672    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 04 20:47:47.600963 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.600944    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 04 20:47:47.601059 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.601045    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 04 20:47:47.601242 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.601226    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 04 20:47:47.601351 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.601263    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4mstc\""
Aug 04 20:47:47.616844 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.616822    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"]
Aug 04 20:47:47.650618 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.650586    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e797d989-4a7d-4f6d-8494-1b1cd9a56292-manifests\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.650758 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.650636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e797d989-4a7d-4f6d-8494-1b1cd9a56292-webhook-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.650758 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.650719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e797d989-4a7d-4f6d-8494-1b1cd9a56292-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.650856 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.650754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsgm2\" (UniqueName: \"kubernetes.io/projected/e797d989-4a7d-4f6d-8494-1b1cd9a56292-kube-api-access-rsgm2\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.650856 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.650799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e797d989-4a7d-4f6d-8494-1b1cd9a56292-tmp\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.751443 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.751412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e797d989-4a7d-4f6d-8494-1b1cd9a56292-webhook-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.751443 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.751451    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e797d989-4a7d-4f6d-8494-1b1cd9a56292-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.751709 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.751472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsgm2\" (UniqueName: \"kubernetes.io/projected/e797d989-4a7d-4f6d-8494-1b1cd9a56292-kube-api-access-rsgm2\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.751709 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.751502    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e797d989-4a7d-4f6d-8494-1b1cd9a56292-tmp\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.751709 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.751535    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e797d989-4a7d-4f6d-8494-1b1cd9a56292-manifests\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.751976 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.751954    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e797d989-4a7d-4f6d-8494-1b1cd9a56292-manifests\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.753940 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.753918    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e797d989-4a7d-4f6d-8494-1b1cd9a56292-tmp\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.754053 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.754036    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e797d989-4a7d-4f6d-8494-1b1cd9a56292-webhook-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.754204 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.754185    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e797d989-4a7d-4f6d-8494-1b1cd9a56292-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.759331 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.759295    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsgm2\" (UniqueName: \"kubernetes.io/projected/e797d989-4a7d-4f6d-8494-1b1cd9a56292-kube-api-access-rsgm2\") pod \"opendatahub-operator-controller-manager-6889885b4f-5bstj\" (UID: \"e797d989-4a7d-4f6d-8494-1b1cd9a56292\") " pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:47.906969 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:47.906941    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:48.074620 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:47:48.074579    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode797d989_4a7d_4f6d_8494_1b1cd9a56292.slice/crio-a39e86a3bf59b84cd92c68510952c2e8c87b44bbd9a35dd32d23bea2180460e2 WatchSource:0}: Error finding container a39e86a3bf59b84cd92c68510952c2e8c87b44bbd9a35dd32d23bea2180460e2: Status 404 returned error can't find the container with id a39e86a3bf59b84cd92c68510952c2e8c87b44bbd9a35dd32d23bea2180460e2
Aug 04 20:47:48.076841 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:48.076814    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"]
Aug 04 20:47:48.682755 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:48.682719    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj" event={"ID":"e797d989-4a7d-4f6d-8494-1b1cd9a56292","Type":"ContainerStarted","Data":"a39e86a3bf59b84cd92c68510952c2e8c87b44bbd9a35dd32d23bea2180460e2"}
Aug 04 20:47:51.696939 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:51.696902    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj" event={"ID":"e797d989-4a7d-4f6d-8494-1b1cd9a56292","Type":"ContainerStarted","Data":"8743049add3040585306317c2be92011368c9aa45d8f5448ff43601a18dd38c6"}
Aug 04 20:47:52.702199 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:52.702169    2563 generic.go:356] "Generic (PLEG): container finished" podID="e797d989-4a7d-4f6d-8494-1b1cd9a56292" containerID="8743049add3040585306317c2be92011368c9aa45d8f5448ff43601a18dd38c6" exitCode=0
Aug 04 20:47:52.702613 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:52.702217    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj" event={"ID":"e797d989-4a7d-4f6d-8494-1b1cd9a56292","Type":"ContainerDied","Data":"8743049add3040585306317c2be92011368c9aa45d8f5448ff43601a18dd38c6"}
Aug 04 20:47:53.709016 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:53.708971    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj" event={"ID":"e797d989-4a7d-4f6d-8494-1b1cd9a56292","Type":"ContainerStarted","Data":"fc576a835c52ad486298859e5271c12de6e5ff1d464afb1f397ca85bc90ff29f"}
Aug 04 20:47:53.709501 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:53.709056    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:47:53.733016 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:47:53.732958    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj" podStartSLOduration=2.63170414 podStartE2EDuration="6.73293941s" totalImagesPullingTime="3.176303429s" totalInitContainerRuntime="924.931841ms" isStatefulPod=false podCreationTimestamp="2026-08-04 20:47:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:47:53.730827445 +0000 UTC m=+420.974446727" watchObservedRunningTime="2026-08-04 20:47:53.73293941 +0000 UTC m=+420.976558693"
Aug 04 20:48:04.715695 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:04.715663    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6889885b4f-5bstj"
Aug 04 20:48:07.969247 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:07.969210    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"]
Aug 04 20:48:07.977307 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:07.977277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"
Aug 04 20:48:07.980050 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:07.980027    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 04 20:48:07.980192 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:07.980177    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 04 20:48:07.980249 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:07.980238    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zxkm5\""
Aug 04 20:48:07.992592 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:07.992553    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"]
Aug 04 20:48:08.111247 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.111210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7zjs\" (UniqueName: \"kubernetes.io/projected/9ad7142d-85b9-4a89-980c-f9367032d5d8-kube-api-access-h7zjs\") pod \"kube-auth-proxy-586b55cbcc-v8lvb\" (UID: \"9ad7142d-85b9-4a89-980c-f9367032d5d8\") " pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"
Aug 04 20:48:08.111247 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.111255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ad7142d-85b9-4a89-980c-f9367032d5d8-tls-certs\") pod \"kube-auth-proxy-586b55cbcc-v8lvb\" (UID: \"9ad7142d-85b9-4a89-980c-f9367032d5d8\") " pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"
Aug 04 20:48:08.111558 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.111347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ad7142d-85b9-4a89-980c-f9367032d5d8-tmp\") pod \"kube-auth-proxy-586b55cbcc-v8lvb\" (UID: \"9ad7142d-85b9-4a89-980c-f9367032d5d8\") " pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"
Aug 04 20:48:08.120229 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.120207    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nxf79"]
Aug 04 20:48:08.123784 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.123764    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nxf79"
Aug 04 20:48:08.126360 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.126300    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tm7kf\""
Aug 04 20:48:08.126360 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.126329    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Aug 04 20:48:08.134844 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.134821    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nxf79"]
Aug 04 20:48:08.212231 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.212201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ad7142d-85b9-4a89-980c-f9367032d5d8-tls-certs\") pod \"kube-auth-proxy-586b55cbcc-v8lvb\" (UID: \"9ad7142d-85b9-4a89-980c-f9367032d5d8\") " pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"
Aug 04 20:48:08.212443 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.212245    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2g5n\" (UniqueName: \"kubernetes.io/projected/ef35ac4c-7176-42bc-9cee-39ada1daab47-kube-api-access-v2g5n\") pod \"odh-model-controller-7998c67dc4-nxf79\" (UID: \"ef35ac4c-7176-42bc-9cee-39ada1daab47\") " pod="opendatahub/odh-model-controller-7998c67dc4-nxf79"
Aug 04 20:48:08.212443 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.212292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ad7142d-85b9-4a89-980c-f9367032d5d8-tmp\") pod \"kube-auth-proxy-586b55cbcc-v8lvb\" (UID: \"9ad7142d-85b9-4a89-980c-f9367032d5d8\") " pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"
Aug 04 20:48:08.212443 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.212341    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef35ac4c-7176-42bc-9cee-39ada1daab47-cert\") pod \"odh-model-controller-7998c67dc4-nxf79\" (UID: \"ef35ac4c-7176-42bc-9cee-39ada1daab47\") " pod="opendatahub/odh-model-controller-7998c67dc4-nxf79"
Aug 04 20:48:08.212443 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.212404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7zjs\" (UniqueName: \"kubernetes.io/projected/9ad7142d-85b9-4a89-980c-f9367032d5d8-kube-api-access-h7zjs\") pod \"kube-auth-proxy-586b55cbcc-v8lvb\" (UID: \"9ad7142d-85b9-4a89-980c-f9367032d5d8\") " pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"
Aug 04 20:48:08.214696 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.214666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ad7142d-85b9-4a89-980c-f9367032d5d8-tmp\") pod \"kube-auth-proxy-586b55cbcc-v8lvb\" (UID: \"9ad7142d-85b9-4a89-980c-f9367032d5d8\") " pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"
Aug 04 20:48:08.214910 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.214889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ad7142d-85b9-4a89-980c-f9367032d5d8-tls-certs\") pod \"kube-auth-proxy-586b55cbcc-v8lvb\" (UID: \"9ad7142d-85b9-4a89-980c-f9367032d5d8\") " pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"
Aug 04 20:48:08.222156 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.222106    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7zjs\" (UniqueName: \"kubernetes.io/projected/9ad7142d-85b9-4a89-980c-f9367032d5d8-kube-api-access-h7zjs\") pod \"kube-auth-proxy-586b55cbcc-v8lvb\" (UID: \"9ad7142d-85b9-4a89-980c-f9367032d5d8\") " pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"
Aug 04 20:48:08.287847 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.287808    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"
Aug 04 20:48:08.313087 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.313057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef35ac4c-7176-42bc-9cee-39ada1daab47-cert\") pod \"odh-model-controller-7998c67dc4-nxf79\" (UID: \"ef35ac4c-7176-42bc-9cee-39ada1daab47\") " pod="opendatahub/odh-model-controller-7998c67dc4-nxf79"
Aug 04 20:48:08.313227 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.313143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2g5n\" (UniqueName: \"kubernetes.io/projected/ef35ac4c-7176-42bc-9cee-39ada1daab47-kube-api-access-v2g5n\") pod \"odh-model-controller-7998c67dc4-nxf79\" (UID: \"ef35ac4c-7176-42bc-9cee-39ada1daab47\") " pod="opendatahub/odh-model-controller-7998c67dc4-nxf79"
Aug 04 20:48:08.313293 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:48:08.313229    2563 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Aug 04 20:48:08.313375 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:48:08.313345    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef35ac4c-7176-42bc-9cee-39ada1daab47-cert podName:ef35ac4c-7176-42bc-9cee-39ada1daab47 nodeName:}" failed. No retries permitted until 2026-08-04 20:48:08.813303974 +0000 UTC m=+436.056923240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ef35ac4c-7176-42bc-9cee-39ada1daab47-cert") pod "odh-model-controller-7998c67dc4-nxf79" (UID: "ef35ac4c-7176-42bc-9cee-39ada1daab47") : secret "odh-model-controller-webhook-cert" not found
Aug 04 20:48:08.326519 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.326483    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2g5n\" (UniqueName: \"kubernetes.io/projected/ef35ac4c-7176-42bc-9cee-39ada1daab47-kube-api-access-v2g5n\") pod \"odh-model-controller-7998c67dc4-nxf79\" (UID: \"ef35ac4c-7176-42bc-9cee-39ada1daab47\") " pod="opendatahub/odh-model-controller-7998c67dc4-nxf79"
Aug 04 20:48:08.428540 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.428515    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb"]
Aug 04 20:48:08.431582 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:48:08.431551    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ad7142d_85b9_4a89_980c_f9367032d5d8.slice/crio-1599fe756da70aef0210f119c3c7d9dfd6b6067e4225e553b039d38a30281c72 WatchSource:0}: Error finding container 1599fe756da70aef0210f119c3c7d9dfd6b6067e4225e553b039d38a30281c72: Status 404 returned error can't find the container with id 1599fe756da70aef0210f119c3c7d9dfd6b6067e4225e553b039d38a30281c72
Aug 04 20:48:08.776173 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.776137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb" event={"ID":"9ad7142d-85b9-4a89-980c-f9367032d5d8","Type":"ContainerStarted","Data":"1599fe756da70aef0210f119c3c7d9dfd6b6067e4225e553b039d38a30281c72"}
Aug 04 20:48:08.817528 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.817496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef35ac4c-7176-42bc-9cee-39ada1daab47-cert\") pod \"odh-model-controller-7998c67dc4-nxf79\" (UID: \"ef35ac4c-7176-42bc-9cee-39ada1daab47\") " pod="opendatahub/odh-model-controller-7998c67dc4-nxf79"
Aug 04 20:48:08.819957 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.819935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef35ac4c-7176-42bc-9cee-39ada1daab47-cert\") pod \"odh-model-controller-7998c67dc4-nxf79\" (UID: \"ef35ac4c-7176-42bc-9cee-39ada1daab47\") " pod="opendatahub/odh-model-controller-7998c67dc4-nxf79"
Aug 04 20:48:08.864099 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.864069    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bf984"]
Aug 04 20:48:08.868692 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.868676    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bf984"
Aug 04 20:48:08.871341 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.871308    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Aug 04 20:48:08.871457 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.871350    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-kkwt8\""
Aug 04 20:48:08.881296 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.881273    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bf984"]
Aug 04 20:48:08.918824 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.918793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9s6g\" (UniqueName: \"kubernetes.io/projected/3c796804-5024-4777-a3f3-ad7efdd8e45c-kube-api-access-n9s6g\") pod \"model-serving-api-54d89bd5fc-bf984\" (UID: \"3c796804-5024-4777-a3f3-ad7efdd8e45c\") " pod="opendatahub/model-serving-api-54d89bd5fc-bf984"
Aug 04 20:48:08.918976 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:08.918853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c796804-5024-4777-a3f3-ad7efdd8e45c-tls-certs\") pod \"model-serving-api-54d89bd5fc-bf984\" (UID: \"3c796804-5024-4777-a3f3-ad7efdd8e45c\") " pod="opendatahub/model-serving-api-54d89bd5fc-bf984"
Aug 04 20:48:09.019392 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.019356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c796804-5024-4777-a3f3-ad7efdd8e45c-tls-certs\") pod \"model-serving-api-54d89bd5fc-bf984\" (UID: \"3c796804-5024-4777-a3f3-ad7efdd8e45c\") " pod="opendatahub/model-serving-api-54d89bd5fc-bf984"
Aug 04 20:48:09.019392 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:48:09.019375    2563 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Aug 04 20:48:09.019803 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:48:09.019436    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c796804-5024-4777-a3f3-ad7efdd8e45c-tls-certs podName:3c796804-5024-4777-a3f3-ad7efdd8e45c nodeName:}" failed. No retries permitted until 2026-08-04 20:48:09.51941839 +0000 UTC m=+436.763037653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3c796804-5024-4777-a3f3-ad7efdd8e45c-tls-certs") pod "model-serving-api-54d89bd5fc-bf984" (UID: "3c796804-5024-4777-a3f3-ad7efdd8e45c") : secret "model-serving-api-tls" not found
Aug 04 20:48:09.019803 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.019463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9s6g\" (UniqueName: \"kubernetes.io/projected/3c796804-5024-4777-a3f3-ad7efdd8e45c-kube-api-access-n9s6g\") pod \"model-serving-api-54d89bd5fc-bf984\" (UID: \"3c796804-5024-4777-a3f3-ad7efdd8e45c\") " pod="opendatahub/model-serving-api-54d89bd5fc-bf984"
Aug 04 20:48:09.028569 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.028516    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9s6g\" (UniqueName: \"kubernetes.io/projected/3c796804-5024-4777-a3f3-ad7efdd8e45c-kube-api-access-n9s6g\") pod \"model-serving-api-54d89bd5fc-bf984\" (UID: \"3c796804-5024-4777-a3f3-ad7efdd8e45c\") " pod="opendatahub/model-serving-api-54d89bd5fc-bf984"
Aug 04 20:48:09.036405 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.036374    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nxf79"
Aug 04 20:48:09.172340 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.172292    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nxf79"]
Aug 04 20:48:09.174492 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:48:09.174462    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef35ac4c_7176_42bc_9cee_39ada1daab47.slice/crio-c162c2189bbd2cb1cd86bfef56a3b73ac21d6c5da7461e20980bbdf373996948 WatchSource:0}: Error finding container c162c2189bbd2cb1cd86bfef56a3b73ac21d6c5da7461e20980bbdf373996948: Status 404 returned error can't find the container with id c162c2189bbd2cb1cd86bfef56a3b73ac21d6c5da7461e20980bbdf373996948
Aug 04 20:48:09.496658 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.495293    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"]
Aug 04 20:48:09.499346 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.498901    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.501711 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.501683    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Aug 04 20:48:09.501816 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.501747    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Aug 04 20:48:09.501871 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.501691    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c82v6\""
Aug 04 20:48:09.502105 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.502089    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Aug 04 20:48:09.502302 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.502287    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Aug 04 20:48:09.513785 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.513763    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"]
Aug 04 20:48:09.526190 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.525648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c796804-5024-4777-a3f3-ad7efdd8e45c-tls-certs\") pod \"model-serving-api-54d89bd5fc-bf984\" (UID: \"3c796804-5024-4777-a3f3-ad7efdd8e45c\") " pod="opendatahub/model-serving-api-54d89bd5fc-bf984"
Aug 04 20:48:09.533107 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.533084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c796804-5024-4777-a3f3-ad7efdd8e45c-tls-certs\") pod \"model-serving-api-54d89bd5fc-bf984\" (UID: \"3c796804-5024-4777-a3f3-ad7efdd8e45c\") " pod="opendatahub/model-serving-api-54d89bd5fc-bf984"
Aug 04 20:48:09.626356 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.626294    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f1abc60-9495-473b-ab2f-5c39e9537dbd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.626548 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.626404    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.626548 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.626453    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.626661 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.626571    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89rpq\" (UniqueName: \"kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-kube-api-access-89rpq\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.626661 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.626617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.626759 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.626694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.626759 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.626721    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.727613 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.727577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89rpq\" (UniqueName: \"kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-kube-api-access-89rpq\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.727803 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.727628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.727803 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.727664    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.727803 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.727685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.727803 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.727719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f1abc60-9495-473b-ab2f-5c39e9537dbd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.727803 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.727766    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.727803 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.727789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.728827 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.728777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.731156 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.731112    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.731606 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.731586    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.732295 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.732266    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.732729 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.732656    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f1abc60-9495-473b-ab2f-5c39e9537dbd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.736241 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.736211    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89rpq\" (UniqueName: \"kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-kube-api-access-89rpq\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.736446 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.736426    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-s98dv\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.778825 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.778735    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bf984"
Aug 04 20:48:09.784156 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.784119    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nxf79" event={"ID":"ef35ac4c-7176-42bc-9cee-39ada1daab47","Type":"ContainerStarted","Data":"c162c2189bbd2cb1cd86bfef56a3b73ac21d6c5da7461e20980bbdf373996948"}
Aug 04 20:48:09.812565 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.812443    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:09.964933 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:09.963424    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bf984"]
Aug 04 20:48:09.968987 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:48:09.968946    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c796804_5024_4777_a3f3_ad7efdd8e45c.slice/crio-be3b59d01b7edc9c193e855986285d0340bc184810f12e2ea5d9d9d249459c14 WatchSource:0}: Error finding container be3b59d01b7edc9c193e855986285d0340bc184810f12e2ea5d9d9d249459c14: Status 404 returned error can't find the container with id be3b59d01b7edc9c193e855986285d0340bc184810f12e2ea5d9d9d249459c14
Aug 04 20:48:10.004190 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:10.003702    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"]
Aug 04 20:48:10.009496 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:48:10.009454    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f1abc60_9495_473b_ab2f_5c39e9537dbd.slice/crio-a429359af41fd42345af1ee22649450f14b9f61cdc57758b30ded31513714e75 WatchSource:0}: Error finding container a429359af41fd42345af1ee22649450f14b9f61cdc57758b30ded31513714e75: Status 404 returned error can't find the container with id a429359af41fd42345af1ee22649450f14b9f61cdc57758b30ded31513714e75
Aug 04 20:48:10.792181 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:10.792103    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv" event={"ID":"8f1abc60-9495-473b-ab2f-5c39e9537dbd","Type":"ContainerStarted","Data":"a429359af41fd42345af1ee22649450f14b9f61cdc57758b30ded31513714e75"}
Aug 04 20:48:10.794064 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:10.793988    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bf984" event={"ID":"3c796804-5024-4777-a3f3-ad7efdd8e45c","Type":"ContainerStarted","Data":"be3b59d01b7edc9c193e855986285d0340bc184810f12e2ea5d9d9d249459c14"}
Aug 04 20:48:14.788694 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:14.788590    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 20:48:14.788694 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:14.788661    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Aug 04 20:48:15.828187 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.828154    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nxf79" event={"ID":"ef35ac4c-7176-42bc-9cee-39ada1daab47","Type":"ContainerStarted","Data":"22c9c852670d1e99b1225e5b6240e90e69257b0b32273cb812400ec4b0c9c928"}
Aug 04 20:48:15.828699 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.828267    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-nxf79"
Aug 04 20:48:15.830065 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.830030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv" event={"ID":"8f1abc60-9495-473b-ab2f-5c39e9537dbd","Type":"ContainerStarted","Data":"3f1737e968e9d2eb2a22796550b0000b2e101b9344f527626a3fe9000e68844e"}
Aug 04 20:48:15.830270 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.830256    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:15.831867 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.831839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb" event={"ID":"9ad7142d-85b9-4a89-980c-f9367032d5d8","Type":"ContainerStarted","Data":"37bd6c8d4b9cef2eb671749b654d80a5ce9c2a0f463e6a03ff0a5a9946e3d969"}
Aug 04 20:48:15.832379 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.832309    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-s98dv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 20:48:15.832505 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.832412    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv" podUID="8f1abc60-9495-473b-ab2f-5c39e9537dbd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 20:48:15.833483 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.833460    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bf984" event={"ID":"3c796804-5024-4777-a3f3-ad7efdd8e45c","Type":"ContainerStarted","Data":"5b35b1bc12d483bccf11b88c61a77a1fce37d4b898c1c3f23096d64dbc520819"}
Aug 04 20:48:15.833608 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.833590    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-bf984"
Aug 04 20:48:15.873072 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.873014    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-nxf79" podStartSLOduration=2.269127974 podStartE2EDuration="7.872997864s" totalImagesPullingTime="5.60386989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:48:15.862973208 +0000 UTC m=+443.106592491" watchObservedRunningTime="2026-08-04 20:48:15.872997864 +0000 UTC m=+443.116617196"
Aug 04 20:48:15.915963 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.915913    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv" podStartSLOduration=2.141570176 podStartE2EDuration="6.915896711s" totalImagesPullingTime="4.774326535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:48:15.913149188 +0000 UTC m=+443.156768470" watchObservedRunningTime="2026-08-04 20:48:15.915896711 +0000 UTC m=+443.159515992"
Aug 04 20:48:15.954215 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.954160    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-bf984" podStartSLOduration=3.144913984 podStartE2EDuration="7.954142653s" totalImagesPullingTime="4.809228669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:48:15.952922912 +0000 UTC m=+443.196542194" watchObservedRunningTime="2026-08-04 20:48:15.954142653 +0000 UTC m=+443.197761934"
Aug 04 20:48:15.980712 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:15.980659    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-586b55cbcc-v8lvb" podStartSLOduration=2.640989127 podStartE2EDuration="8.980642664s" totalImagesPullingTime="6.339653537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:48:15.978665327 +0000 UTC m=+443.222284622" watchObservedRunningTime="2026-08-04 20:48:15.980642664 +0000 UTC m=+443.224261945"
Aug 04 20:48:16.843709 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:16.843679    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:19.726164 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.726125    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"]
Aug 04 20:48:19.729386 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.729364    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"
Aug 04 20:48:19.730178 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.730156    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"]
Aug 04 20:48:19.732245 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.732228    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-7dqlg\""
Aug 04 20:48:19.732606 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.732594    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Aug 04 20:48:19.733039 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.733026    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"
Aug 04 20:48:19.735098 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.735078    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Aug 04 20:48:19.735264 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.735252    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-z6m92\""
Aug 04 20:48:19.744062 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.744044    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"]
Aug 04 20:48:19.747163 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.747146    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"]
Aug 04 20:48:19.818531 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.818500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gszlg\" (UniqueName: \"kubernetes.io/projected/4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a-kube-api-access-gszlg\") pod \"llmisvc-controller-manager-777c6bd7d5-rnkjr\" (UID: \"4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"
Aug 04 20:48:19.818644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.818538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbtgh\" (UniqueName: \"kubernetes.io/projected/cad6b73e-db31-49f4-a41d-02c538737b4d-kube-api-access-rbtgh\") pod \"kserve-controller-manager-d6f4b65b8-5dcrr\" (UID: \"cad6b73e-db31-49f4-a41d-02c538737b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"
Aug 04 20:48:19.818644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.818559    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rnkjr\" (UID: \"4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"
Aug 04 20:48:19.818728 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.818635    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cad6b73e-db31-49f4-a41d-02c538737b4d-cert\") pod \"kserve-controller-manager-d6f4b65b8-5dcrr\" (UID: \"cad6b73e-db31-49f4-a41d-02c538737b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"
Aug 04 20:48:19.919252 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.919228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gszlg\" (UniqueName: \"kubernetes.io/projected/4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a-kube-api-access-gszlg\") pod \"llmisvc-controller-manager-777c6bd7d5-rnkjr\" (UID: \"4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"
Aug 04 20:48:19.919388 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.919258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbtgh\" (UniqueName: \"kubernetes.io/projected/cad6b73e-db31-49f4-a41d-02c538737b4d-kube-api-access-rbtgh\") pod \"kserve-controller-manager-d6f4b65b8-5dcrr\" (UID: \"cad6b73e-db31-49f4-a41d-02c538737b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"
Aug 04 20:48:19.919388 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.919280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rnkjr\" (UID: \"4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"
Aug 04 20:48:19.919388 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.919305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cad6b73e-db31-49f4-a41d-02c538737b4d-cert\") pod \"kserve-controller-manager-d6f4b65b8-5dcrr\" (UID: \"cad6b73e-db31-49f4-a41d-02c538737b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"
Aug 04 20:48:19.919514 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:48:19.919407    2563 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Aug 04 20:48:19.919514 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:48:19.919415    2563 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Aug 04 20:48:19.919514 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:48:19.919471    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cad6b73e-db31-49f4-a41d-02c538737b4d-cert podName:cad6b73e-db31-49f4-a41d-02c538737b4d nodeName:}" failed. No retries permitted until 2026-08-04 20:48:20.419456805 +0000 UTC m=+447.663076066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cad6b73e-db31-49f4-a41d-02c538737b4d-cert") pod "kserve-controller-manager-d6f4b65b8-5dcrr" (UID: "cad6b73e-db31-49f4-a41d-02c538737b4d") : secret "kserve-webhook-server-cert" not found
Aug 04 20:48:19.919514 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:48:19.919487    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a-cert podName:4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a nodeName:}" failed. No retries permitted until 2026-08-04 20:48:20.4194784 +0000 UTC m=+447.663097660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a-cert") pod "llmisvc-controller-manager-777c6bd7d5-rnkjr" (UID: "4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a") : secret "llmisvc-webhook-server-cert" not found
Aug 04 20:48:19.937997 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.937973    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gszlg\" (UniqueName: \"kubernetes.io/projected/4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a-kube-api-access-gszlg\") pod \"llmisvc-controller-manager-777c6bd7d5-rnkjr\" (UID: \"4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"
Aug 04 20:48:19.945449 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:19.945425    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbtgh\" (UniqueName: \"kubernetes.io/projected/cad6b73e-db31-49f4-a41d-02c538737b4d-kube-api-access-rbtgh\") pod \"kserve-controller-manager-d6f4b65b8-5dcrr\" (UID: \"cad6b73e-db31-49f4-a41d-02c538737b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"
Aug 04 20:48:20.423478 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:20.423451    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rnkjr\" (UID: \"4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"
Aug 04 20:48:20.423643 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:20.423494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cad6b73e-db31-49f4-a41d-02c538737b4d-cert\") pod \"kserve-controller-manager-d6f4b65b8-5dcrr\" (UID: \"cad6b73e-db31-49f4-a41d-02c538737b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"
Aug 04 20:48:20.425720 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:20.425696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cad6b73e-db31-49f4-a41d-02c538737b4d-cert\") pod \"kserve-controller-manager-d6f4b65b8-5dcrr\" (UID: \"cad6b73e-db31-49f4-a41d-02c538737b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"
Aug 04 20:48:20.425838 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:20.425802    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rnkjr\" (UID: \"4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"
Aug 04 20:48:20.640182 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:20.640156    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"
Aug 04 20:48:20.647022 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:20.646997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"
Aug 04 20:48:20.781149 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:20.781127    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"]
Aug 04 20:48:20.783659 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:48:20.783632    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad6b73e_db31_49f4_a41d_02c538737b4d.slice/crio-23ccc56614e4f767ecd48785ce750a747745887f2ca387bed436cf6c4f17cbd4 WatchSource:0}: Error finding container 23ccc56614e4f767ecd48785ce750a747745887f2ca387bed436cf6c4f17cbd4: Status 404 returned error can't find the container with id 23ccc56614e4f767ecd48785ce750a747745887f2ca387bed436cf6c4f17cbd4
Aug 04 20:48:20.797498 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:20.797477    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"]
Aug 04 20:48:20.799115 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:48:20.799096    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4d4e8b5c_2b50_46eb_9cbb_06318cd33d1a.slice/crio-88d646fdff6b88f238235f66223852e9dba20ff926bd1a2277c9caf2e813a68d WatchSource:0}: Error finding container 88d646fdff6b88f238235f66223852e9dba20ff926bd1a2277c9caf2e813a68d: Status 404 returned error can't find the container with id 88d646fdff6b88f238235f66223852e9dba20ff926bd1a2277c9caf2e813a68d
Aug 04 20:48:20.858445 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:20.858411    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr" event={"ID":"cad6b73e-db31-49f4-a41d-02c538737b4d","Type":"ContainerStarted","Data":"23ccc56614e4f767ecd48785ce750a747745887f2ca387bed436cf6c4f17cbd4"}
Aug 04 20:48:20.859526 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:20.859500    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr" event={"ID":"4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a","Type":"ContainerStarted","Data":"88d646fdff6b88f238235f66223852e9dba20ff926bd1a2277c9caf2e813a68d"}
Aug 04 20:48:22.282676 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:22.282635    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"]
Aug 04 20:48:22.283067 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:22.282852    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv" podUID="8f1abc60-9495-473b-ab2f-5c39e9537dbd" containerName="discovery" containerID="cri-o://3f1737e968e9d2eb2a22796550b0000b2e101b9344f527626a3fe9000e68844e" gracePeriod=30
Aug 04 20:48:22.872939 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:22.872901    2563 generic.go:356] "Generic (PLEG): container finished" podID="8f1abc60-9495-473b-ab2f-5c39e9537dbd" containerID="3f1737e968e9d2eb2a22796550b0000b2e101b9344f527626a3fe9000e68844e" exitCode=0
Aug 04 20:48:22.873144 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:22.872987    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv" event={"ID":"8f1abc60-9495-473b-ab2f-5c39e9537dbd","Type":"ContainerDied","Data":"3f1737e968e9d2eb2a22796550b0000b2e101b9344f527626a3fe9000e68844e"}
Aug 04 20:48:23.233475 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.233448    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:23.351170 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.351135    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89rpq\" (UniqueName: \"kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-kube-api-access-89rpq\") pod \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") "
Aug 04 20:48:23.351636 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.351186    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-ca-configmap\") pod \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") "
Aug 04 20:48:23.351636 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.351233    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-kubeconfig\") pod \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") "
Aug 04 20:48:23.351636 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.351270    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f1abc60-9495-473b-ab2f-5c39e9537dbd-local-certs\") pod \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") "
Aug 04 20:48:23.351636 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.351297    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-token\") pod \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") "
Aug 04 20:48:23.351636 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.351440    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-cacerts\") pod \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") "
Aug 04 20:48:23.351636 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.351514    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-dns-cert\") pod \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\" (UID: \"8f1abc60-9495-473b-ab2f-5c39e9537dbd\") "
Aug 04 20:48:23.351921 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.351707    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "8f1abc60-9495-473b-ab2f-5c39e9537dbd" (UID: "8f1abc60-9495-473b-ab2f-5c39e9537dbd"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 20:48:23.351921 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.351862    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-ca-configmap\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:48:23.353993 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.353940    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f1abc60-9495-473b-ab2f-5c39e9537dbd-local-certs" (OuterVolumeSpecName: "local-certs") pod "8f1abc60-9495-473b-ab2f-5c39e9537dbd" (UID: "8f1abc60-9495-473b-ab2f-5c39e9537dbd"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:48:23.354108 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.354019    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "8f1abc60-9495-473b-ab2f-5c39e9537dbd" (UID: "8f1abc60-9495-473b-ab2f-5c39e9537dbd"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 20:48:23.354108 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.354027    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f1abc60-9495-473b-ab2f-5c39e9537dbd" (UID: "8f1abc60-9495-473b-ab2f-5c39e9537dbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:48:23.354108 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.354037    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-kube-api-access-89rpq" (OuterVolumeSpecName: "kube-api-access-89rpq") pod "8f1abc60-9495-473b-ab2f-5c39e9537dbd" (UID: "8f1abc60-9495-473b-ab2f-5c39e9537dbd"). InnerVolumeSpecName "kube-api-access-89rpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:48:23.354108 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.354099    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "8f1abc60-9495-473b-ab2f-5c39e9537dbd" (UID: "8f1abc60-9495-473b-ab2f-5c39e9537dbd"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 20:48:23.354261 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.354161    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-cacerts" (OuterVolumeSpecName: "cacerts") pod "8f1abc60-9495-473b-ab2f-5c39e9537dbd" (UID: "8f1abc60-9495-473b-ab2f-5c39e9537dbd"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 20:48:23.452948 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.452881    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-kubeconfig\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:48:23.452948 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.452905    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f1abc60-9495-473b-ab2f-5c39e9537dbd-local-certs\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:48:23.452948 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.452923    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-token\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:48:23.452948 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.452931    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-cacerts\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:48:23.452948 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.452937    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f1abc60-9495-473b-ab2f-5c39e9537dbd-istio-csr-dns-cert\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:48:23.452948 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.452945    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89rpq\" (UniqueName: \"kubernetes.io/projected/8f1abc60-9495-473b-ab2f-5c39e9537dbd-kube-api-access-89rpq\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:48:23.879428 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.879390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv" event={"ID":"8f1abc60-9495-473b-ab2f-5c39e9537dbd","Type":"ContainerDied","Data":"a429359af41fd42345af1ee22649450f14b9f61cdc57758b30ded31513714e75"}
Aug 04 20:48:23.879589 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.879442    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"
Aug 04 20:48:23.879589 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.879466    2563 scope.go:117] "RemoveContainer" containerID="3f1737e968e9d2eb2a22796550b0000b2e101b9344f527626a3fe9000e68844e"
Aug 04 20:48:23.903008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.902982    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"]
Aug 04 20:48:23.909468 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:23.909440    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv"]
Aug 04 20:48:24.886350 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:24.886300    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr" event={"ID":"4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a","Type":"ContainerStarted","Data":"b4c723da2a8038923933652d11b9f51472044ed8a1dfe5c1be3f1d7bfee29a4b"}
Aug 04 20:48:24.886809 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:24.886453    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"
Aug 04 20:48:24.887774 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:24.887751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr" event={"ID":"cad6b73e-db31-49f4-a41d-02c538737b4d","Type":"ContainerStarted","Data":"1925316094f947c0d921ad4613de7b354995ed16ed8dbb24495a7ad99a9dfe41"}
Aug 04 20:48:24.887893 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:24.887823    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"
Aug 04 20:48:24.907363 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:24.907291    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr" podStartSLOduration=2.605366017 podStartE2EDuration="5.907274205s" totalImagesPullingTime="3.301908188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:48:24.90626142 +0000 UTC m=+452.149880694" watchObservedRunningTime="2026-08-04 20:48:24.907274205 +0000 UTC m=+452.150893498"
Aug 04 20:48:24.923658 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:24.923607    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr" podStartSLOduration=2.628388581 podStartE2EDuration="5.923590032s" totalImagesPullingTime="3.295201451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:48:24.922623311 +0000 UTC m=+452.166242616" watchObservedRunningTime="2026-08-04 20:48:24.923590032 +0000 UTC m=+452.167209312"
Aug 04 20:48:25.242451 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:25.242379    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f1abc60-9495-473b-ab2f-5c39e9537dbd" path="/var/lib/kubelet/pods/8f1abc60-9495-473b-ab2f-5c39e9537dbd/volumes"
Aug 04 20:48:26.842927 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:26.842900    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-nxf79"
Aug 04 20:48:26.843351 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:26.843254    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-bf984"
Aug 04 20:48:27.839837 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:27.839805    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-s98dv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.26:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Aug 04 20:48:27.840008 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:27.839861    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s98dv" podUID="8f1abc60-9495-473b-ab2f-5c39e9537dbd" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.26:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Aug 04 20:48:55.895090 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:55.895046    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnkjr"
Aug 04 20:48:55.898057 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:48:55.898039    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5dcrr"
Aug 04 20:49:12.527032 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.526995    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"]
Aug 04 20:49:12.527493 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.527289    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f1abc60-9495-473b-ab2f-5c39e9537dbd" containerName="discovery"
Aug 04 20:49:12.527493 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.527300    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f1abc60-9495-473b-ab2f-5c39e9537dbd" containerName="discovery"
Aug 04 20:49:12.527493 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.527370    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f1abc60-9495-473b-ab2f-5c39e9537dbd" containerName="discovery"
Aug 04 20:49:12.534350 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.534332    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:12.537264 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.537246    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Aug 04 20:49:12.538599 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.538565    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Aug 04 20:49:12.538728 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.538568    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7dxrk\""
Aug 04 20:49:12.540724 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.540697    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"]
Aug 04 20:49:12.616221 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.616187    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:12.616438 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.616230    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:12.616438 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.616256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmxsn\" (UniqueName: \"kubernetes.io/projected/43a98479-cf6d-4b66-89ba-dff03054536c-kube-api-access-jmxsn\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:12.622832 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.622807    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"]
Aug 04 20:49:12.625179 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.625163    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:12.636988 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.636967    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"]
Aug 04 20:49:12.717478 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.717450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:12.717620 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.717483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:12.717620 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.717521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:12.717620 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.717557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:12.717620 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.717583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxsn\" (UniqueName: \"kubernetes.io/projected/43a98479-cf6d-4b66-89ba-dff03054536c-kube-api-access-jmxsn\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:12.717762 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.717623    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mntr\" (UniqueName: \"kubernetes.io/projected/198a30c5-5ab9-4418-b21a-8d0039436bb4-kube-api-access-6mntr\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:12.717842 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.717827    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:12.717876 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.717857    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:12.728537 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.728515    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"]
Aug 04 20:49:12.730480 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.730454    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmxsn\" (UniqueName: \"kubernetes.io/projected/43a98479-cf6d-4b66-89ba-dff03054536c-kube-api-access-jmxsn\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:12.735116 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.735095    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:12.743465 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.743442    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"]
Aug 04 20:49:12.818689 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.818617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:12.818689 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.818654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:12.818689 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.818686    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:12.818934 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.818758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:12.818934 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.818816    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mntr\" (UniqueName: \"kubernetes.io/projected/198a30c5-5ab9-4418-b21a-8d0039436bb4-kube-api-access-6mntr\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:12.818934 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.818840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9z7d\" (UniqueName: \"kubernetes.io/projected/ee4c9727-3272-4f2d-bb5f-f02a176c0021-kube-api-access-j9z7d\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:12.819032 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.818959    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:12.819032 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.819014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:12.829837 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.829807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mntr\" (UniqueName: \"kubernetes.io/projected/198a30c5-5ab9-4418-b21a-8d0039436bb4-kube-api-access-6mntr\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:12.833028 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.833008    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"]
Aug 04 20:49:12.835308 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.835289    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:12.843507 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.843470    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:12.845639 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.845619    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"]
Aug 04 20:49:12.920001 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.919971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdq2g\" (UniqueName: \"kubernetes.io/projected/34f3f6cf-0980-4424-87ec-ca3cfa38776f-kube-api-access-fdq2g\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:12.920158 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.920024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:12.920158 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.920044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:12.920893 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.920194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9z7d\" (UniqueName: \"kubernetes.io/projected/ee4c9727-3272-4f2d-bb5f-f02a176c0021-kube-api-access-j9z7d\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:12.920893 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.920439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:12.920893 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.920485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:12.920893 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.920774    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:12.920893 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.920810    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:12.929930 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.929906    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9z7d\" (UniqueName: \"kubernetes.io/projected/ee4c9727-3272-4f2d-bb5f-f02a176c0021-kube-api-access-j9z7d\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:12.937540 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.937516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:12.983549 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:12.983442    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"]
Aug 04 20:49:12.985827 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:49:12.985800    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a98479_cf6d_4b66_89ba_dff03054536c.slice/crio-a45345bd0ac17488a9bf8bb474d717dc5be8061b0694e655ecdbb1ad82e7f742 WatchSource:0}: Error finding container a45345bd0ac17488a9bf8bb474d717dc5be8061b0694e655ecdbb1ad82e7f742: Status 404 returned error can't find the container with id a45345bd0ac17488a9bf8bb474d717dc5be8061b0694e655ecdbb1ad82e7f742
Aug 04 20:49:13.021230 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.021196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdq2g\" (UniqueName: \"kubernetes.io/projected/34f3f6cf-0980-4424-87ec-ca3cfa38776f-kube-api-access-fdq2g\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:13.021385 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.021251    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:13.021385 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.021289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:13.021609 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.021593    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:13.021671 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.021650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:13.031496 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.031441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdq2g\" (UniqueName: \"kubernetes.io/projected/34f3f6cf-0980-4424-87ec-ca3cfa38776f-kube-api-access-fdq2g\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:13.044979 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.044956    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:13.098666 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.098554    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"]
Aug 04 20:49:13.102990 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:49:13.102950    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198a30c5_5ab9_4418_b21a_8d0039436bb4.slice/crio-31f453cb5a0903ed094173f99f514c91eaa1786f4e1f1846333df5843148bf7f WatchSource:0}: Error finding container 31f453cb5a0903ed094173f99f514c91eaa1786f4e1f1846333df5843148bf7f: Status 404 returned error can't find the container with id 31f453cb5a0903ed094173f99f514c91eaa1786f4e1f1846333df5843148bf7f
Aug 04 20:49:13.112551 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.112350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg" event={"ID":"43a98479-cf6d-4b66-89ba-dff03054536c","Type":"ContainerStarted","Data":"a073031020be5645a3228b19155f043efa011b7497eb20dfa2dcf0ceaa5326ed"}
Aug 04 20:49:13.112551 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.112419    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg" event={"ID":"43a98479-cf6d-4b66-89ba-dff03054536c","Type":"ContainerStarted","Data":"a45345bd0ac17488a9bf8bb474d717dc5be8061b0694e655ecdbb1ad82e7f742"}
Aug 04 20:49:13.144565 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.144355    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:13.201131 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.200105    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"]
Aug 04 20:49:13.211111 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:49:13.208673    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee4c9727_3272_4f2d_bb5f_f02a176c0021.slice/crio-1115908a0d3239d6d1f26891637fe36eb085323946e70a8468fe4133811b8e44 WatchSource:0}: Error finding container 1115908a0d3239d6d1f26891637fe36eb085323946e70a8468fe4133811b8e44: Status 404 returned error can't find the container with id 1115908a0d3239d6d1f26891637fe36eb085323946e70a8468fe4133811b8e44
Aug 04 20:49:13.300880 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:13.300854    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"]
Aug 04 20:49:13.375146 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:49:13.375121    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34f3f6cf_0980_4424_87ec_ca3cfa38776f.slice/crio-07ad4bf84241e3a0300859b1291de1ca1b249027b2019bee2560253d976572ba WatchSource:0}: Error finding container 07ad4bf84241e3a0300859b1291de1ca1b249027b2019bee2560253d976572ba: Status 404 returned error can't find the container with id 07ad4bf84241e3a0300859b1291de1ca1b249027b2019bee2560253d976572ba
Aug 04 20:49:14.119116 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:14.119079    2563 generic.go:356] "Generic (PLEG): container finished" podID="ee4c9727-3272-4f2d-bb5f-f02a176c0021" containerID="0321d267842f061100dcddced348ce0c09f66fce365ee4f257f50ec1903471ef" exitCode=0
Aug 04 20:49:14.119548 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:14.119151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz" event={"ID":"ee4c9727-3272-4f2d-bb5f-f02a176c0021","Type":"ContainerDied","Data":"0321d267842f061100dcddced348ce0c09f66fce365ee4f257f50ec1903471ef"}
Aug 04 20:49:14.119548 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:14.119202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz" event={"ID":"ee4c9727-3272-4f2d-bb5f-f02a176c0021","Type":"ContainerStarted","Data":"1115908a0d3239d6d1f26891637fe36eb085323946e70a8468fe4133811b8e44"}
Aug 04 20:49:14.120901 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:14.120881    2563 generic.go:356] "Generic (PLEG): container finished" podID="198a30c5-5ab9-4418-b21a-8d0039436bb4" containerID="a307221e25dcba342a7ba36b49c6727c48e84fc17dfa12966121acbd3b611f6c" exitCode=0
Aug 04 20:49:14.121056 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:14.121024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4" event={"ID":"198a30c5-5ab9-4418-b21a-8d0039436bb4","Type":"ContainerDied","Data":"a307221e25dcba342a7ba36b49c6727c48e84fc17dfa12966121acbd3b611f6c"}
Aug 04 20:49:14.121165 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:14.121067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4" event={"ID":"198a30c5-5ab9-4418-b21a-8d0039436bb4","Type":"ContainerStarted","Data":"31f453cb5a0903ed094173f99f514c91eaa1786f4e1f1846333df5843148bf7f"}
Aug 04 20:49:14.122678 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:14.122655    2563 generic.go:356] "Generic (PLEG): container finished" podID="34f3f6cf-0980-4424-87ec-ca3cfa38776f" containerID="fd2e75ad815d0662ff6e5f22c7e56afa87d181c9b823243be5a0645b243dec51" exitCode=0
Aug 04 20:49:14.122804 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:14.122736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b" event={"ID":"34f3f6cf-0980-4424-87ec-ca3cfa38776f","Type":"ContainerDied","Data":"fd2e75ad815d0662ff6e5f22c7e56afa87d181c9b823243be5a0645b243dec51"}
Aug 04 20:49:14.122804 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:14.122780    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b" event={"ID":"34f3f6cf-0980-4424-87ec-ca3cfa38776f","Type":"ContainerStarted","Data":"07ad4bf84241e3a0300859b1291de1ca1b249027b2019bee2560253d976572ba"}
Aug 04 20:49:14.124187 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:14.124169    2563 generic.go:356] "Generic (PLEG): container finished" podID="43a98479-cf6d-4b66-89ba-dff03054536c" containerID="a073031020be5645a3228b19155f043efa011b7497eb20dfa2dcf0ceaa5326ed" exitCode=0
Aug 04 20:49:14.124256 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:14.124201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg" event={"ID":"43a98479-cf6d-4b66-89ba-dff03054536c","Type":"ContainerDied","Data":"a073031020be5645a3228b19155f043efa011b7497eb20dfa2dcf0ceaa5326ed"}
Aug 04 20:49:15.132854 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:15.132827    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg" event={"ID":"43a98479-cf6d-4b66-89ba-dff03054536c","Type":"ContainerStarted","Data":"8dcdcb255aee91dffd70d75d12fbc19ea2321dca45ad7ba2059a1b48ef696aba"}
Aug 04 20:49:15.135647 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:15.135616    2563 generic.go:356] "Generic (PLEG): container finished" podID="ee4c9727-3272-4f2d-bb5f-f02a176c0021" containerID="30662718c403ca10ef4ce47e796eb6de95cc037843269fc054081e80f4420f4e" exitCode=0
Aug 04 20:49:15.135759 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:15.135712    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz" event={"ID":"ee4c9727-3272-4f2d-bb5f-f02a176c0021","Type":"ContainerDied","Data":"30662718c403ca10ef4ce47e796eb6de95cc037843269fc054081e80f4420f4e"}
Aug 04 20:49:16.147725 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:16.147692    2563 generic.go:356] "Generic (PLEG): container finished" podID="43a98479-cf6d-4b66-89ba-dff03054536c" containerID="8dcdcb255aee91dffd70d75d12fbc19ea2321dca45ad7ba2059a1b48ef696aba" exitCode=0
Aug 04 20:49:16.148175 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:16.147770    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg" event={"ID":"43a98479-cf6d-4b66-89ba-dff03054536c","Type":"ContainerDied","Data":"8dcdcb255aee91dffd70d75d12fbc19ea2321dca45ad7ba2059a1b48ef696aba"}
Aug 04 20:49:16.149840 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:16.149821    2563 generic.go:356] "Generic (PLEG): container finished" podID="ee4c9727-3272-4f2d-bb5f-f02a176c0021" containerID="851949a136265a4d2ad6458522e52e5d98910b566d91f57a3cb0e821bb85d37d" exitCode=0
Aug 04 20:49:16.149933 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:16.149883    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz" event={"ID":"ee4c9727-3272-4f2d-bb5f-f02a176c0021","Type":"ContainerDied","Data":"851949a136265a4d2ad6458522e52e5d98910b566d91f57a3cb0e821bb85d37d"}
Aug 04 20:49:16.151457 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:16.151439    2563 generic.go:356] "Generic (PLEG): container finished" podID="198a30c5-5ab9-4418-b21a-8d0039436bb4" containerID="aeacc84dc9a82e0034a59a963e5979fb0a434c39356d5fedb467a26d36383f29" exitCode=0
Aug 04 20:49:16.151542 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:16.151510    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4" event={"ID":"198a30c5-5ab9-4418-b21a-8d0039436bb4","Type":"ContainerDied","Data":"aeacc84dc9a82e0034a59a963e5979fb0a434c39356d5fedb467a26d36383f29"}
Aug 04 20:49:16.153377 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:16.153355    2563 generic.go:356] "Generic (PLEG): container finished" podID="34f3f6cf-0980-4424-87ec-ca3cfa38776f" containerID="904552fc941dfc45f88ea30a9759baf7b3bccdd74f12a79744e6e15d59783d6b" exitCode=0
Aug 04 20:49:16.153482 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:16.153399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b" event={"ID":"34f3f6cf-0980-4424-87ec-ca3cfa38776f","Type":"ContainerDied","Data":"904552fc941dfc45f88ea30a9759baf7b3bccdd74f12a79744e6e15d59783d6b"}
Aug 04 20:49:17.159668 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.159641    2563 generic.go:356] "Generic (PLEG): container finished" podID="198a30c5-5ab9-4418-b21a-8d0039436bb4" containerID="b9ddb2ed0c4baa57b1d9403a335db87d7b9720def0b94219fa43e982ac90adc4" exitCode=0
Aug 04 20:49:17.160135 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.159719    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4" event={"ID":"198a30c5-5ab9-4418-b21a-8d0039436bb4","Type":"ContainerDied","Data":"b9ddb2ed0c4baa57b1d9403a335db87d7b9720def0b94219fa43e982ac90adc4"}
Aug 04 20:49:17.161585 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.161564    2563 generic.go:356] "Generic (PLEG): container finished" podID="34f3f6cf-0980-4424-87ec-ca3cfa38776f" containerID="0d1da8baafb02a3c2d8941752e3b0d434b3e505ea12c1893f0765951e87f3671" exitCode=0
Aug 04 20:49:17.161698 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.161640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b" event={"ID":"34f3f6cf-0980-4424-87ec-ca3cfa38776f","Type":"ContainerDied","Data":"0d1da8baafb02a3c2d8941752e3b0d434b3e505ea12c1893f0765951e87f3671"}
Aug 04 20:49:17.163499 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.163482    2563 generic.go:356] "Generic (PLEG): container finished" podID="43a98479-cf6d-4b66-89ba-dff03054536c" containerID="f36c21051d7048755d7f9250f36192920e5238a38426e28906aabf7f37fffb2a" exitCode=0
Aug 04 20:49:17.163596 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.163513    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg" event={"ID":"43a98479-cf6d-4b66-89ba-dff03054536c","Type":"ContainerDied","Data":"f36c21051d7048755d7f9250f36192920e5238a38426e28906aabf7f37fffb2a"}
Aug 04 20:49:17.292369 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.292346    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:17.358988 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.358958    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-bundle\") pod \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") "
Aug 04 20:49:17.359158 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.359025    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-util\") pod \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") "
Aug 04 20:49:17.359158 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.359054    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9z7d\" (UniqueName: \"kubernetes.io/projected/ee4c9727-3272-4f2d-bb5f-f02a176c0021-kube-api-access-j9z7d\") pod \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\" (UID: \"ee4c9727-3272-4f2d-bb5f-f02a176c0021\") "
Aug 04 20:49:17.359452 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.359429    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-bundle" (OuterVolumeSpecName: "bundle") pod "ee4c9727-3272-4f2d-bb5f-f02a176c0021" (UID: "ee4c9727-3272-4f2d-bb5f-f02a176c0021"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:49:17.361232 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.361208    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee4c9727-3272-4f2d-bb5f-f02a176c0021-kube-api-access-j9z7d" (OuterVolumeSpecName: "kube-api-access-j9z7d") pod "ee4c9727-3272-4f2d-bb5f-f02a176c0021" (UID: "ee4c9727-3272-4f2d-bb5f-f02a176c0021"). InnerVolumeSpecName "kube-api-access-j9z7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:49:17.363795 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.363774    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-util" (OuterVolumeSpecName: "util") pod "ee4c9727-3272-4f2d-bb5f-f02a176c0021" (UID: "ee4c9727-3272-4f2d-bb5f-f02a176c0021"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:49:17.460490 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.460390    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-bundle\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:17.460490 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.460445    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee4c9727-3272-4f2d-bb5f-f02a176c0021-util\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:17.460490 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:17.460455    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9z7d\" (UniqueName: \"kubernetes.io/projected/ee4c9727-3272-4f2d-bb5f-f02a176c0021-kube-api-access-j9z7d\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:18.170449 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.170418    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz"
Aug 04 20:49:18.170818 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.170447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6csz" event={"ID":"ee4c9727-3272-4f2d-bb5f-f02a176c0021","Type":"ContainerDied","Data":"1115908a0d3239d6d1f26891637fe36eb085323946e70a8468fe4133811b8e44"}
Aug 04 20:49:18.170818 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.170495    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1115908a0d3239d6d1f26891637fe36eb085323946e70a8468fe4133811b8e44"
Aug 04 20:49:18.318558 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.318536    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:18.321484 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.321464    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:18.352194 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.352169    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:18.367630 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.367604    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-util\") pod \"198a30c5-5ab9-4418-b21a-8d0039436bb4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") "
Aug 04 20:49:18.367769 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.367669    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-bundle\") pod \"198a30c5-5ab9-4418-b21a-8d0039436bb4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") "
Aug 04 20:49:18.367769 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.367703    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-bundle\") pod \"43a98479-cf6d-4b66-89ba-dff03054536c\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") "
Aug 04 20:49:18.367769 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.367756    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmxsn\" (UniqueName: \"kubernetes.io/projected/43a98479-cf6d-4b66-89ba-dff03054536c-kube-api-access-jmxsn\") pod \"43a98479-cf6d-4b66-89ba-dff03054536c\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") "
Aug 04 20:49:18.367876 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.367791    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mntr\" (UniqueName: \"kubernetes.io/projected/198a30c5-5ab9-4418-b21a-8d0039436bb4-kube-api-access-6mntr\") pod \"198a30c5-5ab9-4418-b21a-8d0039436bb4\" (UID: \"198a30c5-5ab9-4418-b21a-8d0039436bb4\") "
Aug 04 20:49:18.367876 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.367834    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-util\") pod \"43a98479-cf6d-4b66-89ba-dff03054536c\" (UID: \"43a98479-cf6d-4b66-89ba-dff03054536c\") "
Aug 04 20:49:18.368709 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.368368    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-bundle" (OuterVolumeSpecName: "bundle") pod "43a98479-cf6d-4b66-89ba-dff03054536c" (UID: "43a98479-cf6d-4b66-89ba-dff03054536c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:49:18.368709 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.368484    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-bundle\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:18.368709 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.368606    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-bundle" (OuterVolumeSpecName: "bundle") pod "198a30c5-5ab9-4418-b21a-8d0039436bb4" (UID: "198a30c5-5ab9-4418-b21a-8d0039436bb4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:49:18.370224 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.370192    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/198a30c5-5ab9-4418-b21a-8d0039436bb4-kube-api-access-6mntr" (OuterVolumeSpecName: "kube-api-access-6mntr") pod "198a30c5-5ab9-4418-b21a-8d0039436bb4" (UID: "198a30c5-5ab9-4418-b21a-8d0039436bb4"). InnerVolumeSpecName "kube-api-access-6mntr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:49:18.370304 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.370223    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43a98479-cf6d-4b66-89ba-dff03054536c-kube-api-access-jmxsn" (OuterVolumeSpecName: "kube-api-access-jmxsn") pod "43a98479-cf6d-4b66-89ba-dff03054536c" (UID: "43a98479-cf6d-4b66-89ba-dff03054536c"). InnerVolumeSpecName "kube-api-access-jmxsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:49:18.469460 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.469377    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-bundle\") pod \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") "
Aug 04 20:49:18.469460 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.469446    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdq2g\" (UniqueName: \"kubernetes.io/projected/34f3f6cf-0980-4424-87ec-ca3cfa38776f-kube-api-access-fdq2g\") pod \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") "
Aug 04 20:49:18.469670 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.469475    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-util\") pod \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\" (UID: \"34f3f6cf-0980-4424-87ec-ca3cfa38776f\") "
Aug 04 20:49:18.469670 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.469625    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmxsn\" (UniqueName: \"kubernetes.io/projected/43a98479-cf6d-4b66-89ba-dff03054536c-kube-api-access-jmxsn\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:18.469670 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.469635    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mntr\" (UniqueName: \"kubernetes.io/projected/198a30c5-5ab9-4418-b21a-8d0039436bb4-kube-api-access-6mntr\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:18.469670 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.469644    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-bundle\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:18.469928 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.469903    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-bundle" (OuterVolumeSpecName: "bundle") pod "34f3f6cf-0980-4424-87ec-ca3cfa38776f" (UID: "34f3f6cf-0980-4424-87ec-ca3cfa38776f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:49:18.471634 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.471614    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34f3f6cf-0980-4424-87ec-ca3cfa38776f-kube-api-access-fdq2g" (OuterVolumeSpecName: "kube-api-access-fdq2g") pod "34f3f6cf-0980-4424-87ec-ca3cfa38776f" (UID: "34f3f6cf-0980-4424-87ec-ca3cfa38776f"). InnerVolumeSpecName "kube-api-access-fdq2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:49:18.474476 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.474455    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-util" (OuterVolumeSpecName: "util") pod "34f3f6cf-0980-4424-87ec-ca3cfa38776f" (UID: "34f3f6cf-0980-4424-87ec-ca3cfa38776f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:49:18.570156 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.570116    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdq2g\" (UniqueName: \"kubernetes.io/projected/34f3f6cf-0980-4424-87ec-ca3cfa38776f-kube-api-access-fdq2g\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:18.570156 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.570147    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-util\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:18.570156 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.570157    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34f3f6cf-0980-4424-87ec-ca3cfa38776f-bundle\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:18.579971 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.579941    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-util" (OuterVolumeSpecName: "util") pod "198a30c5-5ab9-4418-b21a-8d0039436bb4" (UID: "198a30c5-5ab9-4418-b21a-8d0039436bb4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:49:18.580470 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.580450    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-util" (OuterVolumeSpecName: "util") pod "43a98479-cf6d-4b66-89ba-dff03054536c" (UID: "43a98479-cf6d-4b66-89ba-dff03054536c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:49:18.671038 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.671005    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43a98479-cf6d-4b66-89ba-dff03054536c-util\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:18.671038 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:18.671031    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/198a30c5-5ab9-4418-b21a-8d0039436bb4-util\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:19.178469 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:19.178440    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b"
Aug 04 20:49:19.178469 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:19.178446    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a09l64b" event={"ID":"34f3f6cf-0980-4424-87ec-ca3cfa38776f","Type":"ContainerDied","Data":"07ad4bf84241e3a0300859b1291de1ca1b249027b2019bee2560253d976572ba"}
Aug 04 20:49:19.178966 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:19.178485    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07ad4bf84241e3a0300859b1291de1ca1b249027b2019bee2560253d976572ba"
Aug 04 20:49:19.180344 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:19.180307    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg"
Aug 04 20:49:19.180344 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:19.180333    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblfjlg" event={"ID":"43a98479-cf6d-4b66-89ba-dff03054536c","Type":"ContainerDied","Data":"a45345bd0ac17488a9bf8bb474d717dc5be8061b0694e655ecdbb1ad82e7f742"}
Aug 04 20:49:19.180504 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:19.180364    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a45345bd0ac17488a9bf8bb474d717dc5be8061b0694e655ecdbb1ad82e7f742"
Aug 04 20:49:19.182154 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:19.182131    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4" event={"ID":"198a30c5-5ab9-4418-b21a-8d0039436bb4","Type":"ContainerDied","Data":"31f453cb5a0903ed094173f99f514c91eaa1786f4e1f1846333df5843148bf7f"}
Aug 04 20:49:19.182154 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:19.182150    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jwbw4"
Aug 04 20:49:19.182340 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:19.182170    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31f453cb5a0903ed094173f99f514c91eaa1786f4e1f1846333df5843148bf7f"
Aug 04 20:49:24.847219 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847173    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5"]
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847480    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43a98479-cf6d-4b66-89ba-dff03054536c" containerName="extract"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847491    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="43a98479-cf6d-4b66-89ba-dff03054536c" containerName="extract"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847500    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="198a30c5-5ab9-4418-b21a-8d0039436bb4" containerName="extract"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847507    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="198a30c5-5ab9-4418-b21a-8d0039436bb4" containerName="extract"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847517    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34f3f6cf-0980-4424-87ec-ca3cfa38776f" containerName="util"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847523    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34f3f6cf-0980-4424-87ec-ca3cfa38776f" containerName="util"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847528    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="198a30c5-5ab9-4418-b21a-8d0039436bb4" containerName="pull"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847533    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="198a30c5-5ab9-4418-b21a-8d0039436bb4" containerName="pull"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847543    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34f3f6cf-0980-4424-87ec-ca3cfa38776f" containerName="extract"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847548    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34f3f6cf-0980-4424-87ec-ca3cfa38776f" containerName="extract"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847555    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee4c9727-3272-4f2d-bb5f-f02a176c0021" containerName="pull"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847559    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee4c9727-3272-4f2d-bb5f-f02a176c0021" containerName="pull"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847565    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34f3f6cf-0980-4424-87ec-ca3cfa38776f" containerName="pull"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847569    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34f3f6cf-0980-4424-87ec-ca3cfa38776f" containerName="pull"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847574    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee4c9727-3272-4f2d-bb5f-f02a176c0021" containerName="extract"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847578    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee4c9727-3272-4f2d-bb5f-f02a176c0021" containerName="extract"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847583    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="198a30c5-5ab9-4418-b21a-8d0039436bb4" containerName="util"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847588    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="198a30c5-5ab9-4418-b21a-8d0039436bb4" containerName="util"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847593    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee4c9727-3272-4f2d-bb5f-f02a176c0021" containerName="util"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847598    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee4c9727-3272-4f2d-bb5f-f02a176c0021" containerName="util"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847604    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43a98479-cf6d-4b66-89ba-dff03054536c" containerName="util"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847609    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="43a98479-cf6d-4b66-89ba-dff03054536c" containerName="util"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847613    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43a98479-cf6d-4b66-89ba-dff03054536c" containerName="pull"
Aug 04 20:49:24.847644 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847618    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="43a98479-cf6d-4b66-89ba-dff03054536c" containerName="pull"
Aug 04 20:49:24.848271 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847665    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="43a98479-cf6d-4b66-89ba-dff03054536c" containerName="extract"
Aug 04 20:49:24.848271 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847672    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="198a30c5-5ab9-4418-b21a-8d0039436bb4" containerName="extract"
Aug 04 20:49:24.848271 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847679    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="34f3f6cf-0980-4424-87ec-ca3cfa38776f" containerName="extract"
Aug 04 20:49:24.848271 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.847685    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee4c9727-3272-4f2d-bb5f-f02a176c0021" containerName="extract"
Aug 04 20:49:24.852535 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.852518    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5"
Aug 04 20:49:24.856674 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.856652    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Aug 04 20:49:24.856772 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.856672    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 04 20:49:24.856772 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.856702    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 04 20:49:24.863403 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.863385    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dvt8q\""
Aug 04 20:49:24.884827 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.884799    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5"]
Aug 04 20:49:24.922993 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:24.922956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jkjb\" (UniqueName: \"kubernetes.io/projected/cfbc6f6f-3db4-4083-9ca2-3814c9124571-kube-api-access-4jkjb\") pod \"dns-operator-controller-manager-5985564776-w2lr5\" (UID: \"cfbc6f6f-3db4-4083-9ca2-3814c9124571\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5"
Aug 04 20:49:25.023849 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:25.023819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jkjb\" (UniqueName: \"kubernetes.io/projected/cfbc6f6f-3db4-4083-9ca2-3814c9124571-kube-api-access-4jkjb\") pod \"dns-operator-controller-manager-5985564776-w2lr5\" (UID: \"cfbc6f6f-3db4-4083-9ca2-3814c9124571\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5"
Aug 04 20:49:25.042082 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:25.042055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jkjb\" (UniqueName: \"kubernetes.io/projected/cfbc6f6f-3db4-4083-9ca2-3814c9124571-kube-api-access-4jkjb\") pod \"dns-operator-controller-manager-5985564776-w2lr5\" (UID: \"cfbc6f6f-3db4-4083-9ca2-3814c9124571\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5"
Aug 04 20:49:25.162212 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:25.162175    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5"
Aug 04 20:49:25.309523 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:25.309471    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5"]
Aug 04 20:49:25.311775 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:49:25.311748    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbc6f6f_3db4_4083_9ca2_3814c9124571.slice/crio-b08de243d7cf94a2500d1871bbc0122ff7ff636c1b63b8a0cbe274424a6538a1 WatchSource:0}: Error finding container b08de243d7cf94a2500d1871bbc0122ff7ff636c1b63b8a0cbe274424a6538a1: Status 404 returned error can't find the container with id b08de243d7cf94a2500d1871bbc0122ff7ff636c1b63b8a0cbe274424a6538a1
Aug 04 20:49:26.218804 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:26.218761    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5" event={"ID":"cfbc6f6f-3db4-4083-9ca2-3814c9124571","Type":"ContainerStarted","Data":"b08de243d7cf94a2500d1871bbc0122ff7ff636c1b63b8a0cbe274424a6538a1"}
Aug 04 20:49:28.230021 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:28.229984    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5" event={"ID":"cfbc6f6f-3db4-4083-9ca2-3814c9124571","Type":"ContainerStarted","Data":"448fc00cd2f684899d1fa970a1eee4079812c206d616123dcb4512af34e78975"}
Aug 04 20:49:28.230502 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:28.230092    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5"
Aug 04 20:49:28.259828 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:28.259774    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5" podStartSLOduration=1.8591728600000001 podStartE2EDuration="4.259759023s" totalImagesPullingTime="2.400586163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:49:28.257702643 +0000 UTC m=+515.501321925" watchObservedRunningTime="2026-08-04 20:49:28.259759023 +0000 UTC m=+515.503378307"
Aug 04 20:49:29.428480 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.428448    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"]
Aug 04 20:49:29.431794 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.431774    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:29.435175 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.435157    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n66wc\""
Aug 04 20:49:29.451983 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.451953    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"]
Aug 04 20:49:29.563035 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.562999    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" (UID: \"2a2c4492-8eca-45bc-a03c-f04d0c4631e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:29.563035 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.563036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76q5l\" (UniqueName: \"kubernetes.io/projected/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-kube-api-access-76q5l\") pod \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" (UID: \"2a2c4492-8eca-45bc-a03c-f04d0c4631e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:29.663724 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.663683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" (UID: \"2a2c4492-8eca-45bc-a03c-f04d0c4631e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:29.663724 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.663724    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76q5l\" (UniqueName: \"kubernetes.io/projected/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-kube-api-access-76q5l\") pod \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" (UID: \"2a2c4492-8eca-45bc-a03c-f04d0c4631e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:29.664106 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.664086    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" (UID: \"2a2c4492-8eca-45bc-a03c-f04d0c4631e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:29.675301 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.675269    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76q5l\" (UniqueName: \"kubernetes.io/projected/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-kube-api-access-76q5l\") pod \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" (UID: \"2a2c4492-8eca-45bc-a03c-f04d0c4631e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:29.741391 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.741281    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:29.886930 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:29.886896    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"]
Aug 04 20:49:29.889874 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:49:29.889849    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2c4492_8eca_45bc_a03c_f04d0c4631e1.slice/crio-0dfcb591ba6986f37cd835a8fcd9e578696ad93af05a8c88d6262a37e5dd4c25 WatchSource:0}: Error finding container 0dfcb591ba6986f37cd835a8fcd9e578696ad93af05a8c88d6262a37e5dd4c25: Status 404 returned error can't find the container with id 0dfcb591ba6986f37cd835a8fcd9e578696ad93af05a8c88d6262a37e5dd4c25
Aug 04 20:49:30.244848 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:30.244812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m" event={"ID":"2a2c4492-8eca-45bc-a03c-f04d0c4631e1","Type":"ContainerStarted","Data":"0dfcb591ba6986f37cd835a8fcd9e578696ad93af05a8c88d6262a37e5dd4c25"}
Aug 04 20:49:31.482639 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:31.482606    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"]
Aug 04 20:49:31.485848 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:31.485830    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:31.494982 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:31.494961    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-crjhv\""
Aug 04 20:49:31.508953 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:31.508923    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"]
Aug 04 20:49:31.578543 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:31.578507    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h8k7\" (UniqueName: \"kubernetes.io/projected/5552316b-cfc8-4c29-b82b-03ffe1fa6e91-kube-api-access-6h8k7\") pod \"limitador-operator-controller-manager-6c549b6d85-qcgvv\" (UID: \"5552316b-cfc8-4c29-b82b-03ffe1fa6e91\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:31.679982 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:31.679943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h8k7\" (UniqueName: \"kubernetes.io/projected/5552316b-cfc8-4c29-b82b-03ffe1fa6e91-kube-api-access-6h8k7\") pod \"limitador-operator-controller-manager-6c549b6d85-qcgvv\" (UID: \"5552316b-cfc8-4c29-b82b-03ffe1fa6e91\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:31.691310 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:31.691285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h8k7\" (UniqueName: \"kubernetes.io/projected/5552316b-cfc8-4c29-b82b-03ffe1fa6e91-kube-api-access-6h8k7\") pod \"limitador-operator-controller-manager-6c549b6d85-qcgvv\" (UID: \"5552316b-cfc8-4c29-b82b-03ffe1fa6e91\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:31.796232 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:31.796152    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:31.967245 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:31.967221    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"]
Aug 04 20:49:31.969858 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:49:31.969829    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5552316b_cfc8_4c29_b82b_03ffe1fa6e91.slice/crio-b49d85a4b9b2a7472ea063acc864e5270c883ddd452b9e5195f253063274e980 WatchSource:0}: Error finding container b49d85a4b9b2a7472ea063acc864e5270c883ddd452b9e5195f253063274e980: Status 404 returned error can't find the container with id b49d85a4b9b2a7472ea063acc864e5270c883ddd452b9e5195f253063274e980
Aug 04 20:49:32.256700 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:32.256665    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv" event={"ID":"5552316b-cfc8-4c29-b82b-03ffe1fa6e91","Type":"ContainerStarted","Data":"b49d85a4b9b2a7472ea063acc864e5270c883ddd452b9e5195f253063274e980"}
Aug 04 20:49:34.315974 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:34.315942    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-jfjfl"]
Aug 04 20:49:34.319475 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:34.319451    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-jfjfl"
Aug 04 20:49:34.322468 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:34.322448    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-tts2k\""
Aug 04 20:49:34.333250 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:34.333228    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-jfjfl"]
Aug 04 20:49:34.418384 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:34.418353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8hqb\" (UniqueName: \"kubernetes.io/projected/03a04462-0175-400e-9968-f5f71987f040-kube-api-access-c8hqb\") pod \"authorino-operator-68d88948d8-jfjfl\" (UID: \"03a04462-0175-400e-9968-f5f71987f040\") " pod="kuadrant-system/authorino-operator-68d88948d8-jfjfl"
Aug 04 20:49:34.518871 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:34.518828    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8hqb\" (UniqueName: \"kubernetes.io/projected/03a04462-0175-400e-9968-f5f71987f040-kube-api-access-c8hqb\") pod \"authorino-operator-68d88948d8-jfjfl\" (UID: \"03a04462-0175-400e-9968-f5f71987f040\") " pod="kuadrant-system/authorino-operator-68d88948d8-jfjfl"
Aug 04 20:49:34.531675 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:34.531646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8hqb\" (UniqueName: \"kubernetes.io/projected/03a04462-0175-400e-9968-f5f71987f040-kube-api-access-c8hqb\") pod \"authorino-operator-68d88948d8-jfjfl\" (UID: \"03a04462-0175-400e-9968-f5f71987f040\") " pod="kuadrant-system/authorino-operator-68d88948d8-jfjfl"
Aug 04 20:49:34.632969 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:34.632932    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-jfjfl"
Aug 04 20:49:35.035553 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:35.035444    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-jfjfl"]
Aug 04 20:49:35.038169 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:49:35.038141    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a04462_0175_400e_9968_f5f71987f040.slice/crio-ef09ccf49cca088019663291f6171a7d5734fd5ed7e1f8582b1fe9d826a444da WatchSource:0}: Error finding container ef09ccf49cca088019663291f6171a7d5734fd5ed7e1f8582b1fe9d826a444da: Status 404 returned error can't find the container with id ef09ccf49cca088019663291f6171a7d5734fd5ed7e1f8582b1fe9d826a444da
Aug 04 20:49:35.275458 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:35.275343    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m" event={"ID":"2a2c4492-8eca-45bc-a03c-f04d0c4631e1","Type":"ContainerStarted","Data":"0f6b36a2beb71015b2f54e042c2aa68f62c0b25f663d90c930bf2807816801d1"}
Aug 04 20:49:35.275458 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:35.275421    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:35.276874 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:35.276847    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv" event={"ID":"5552316b-cfc8-4c29-b82b-03ffe1fa6e91","Type":"ContainerStarted","Data":"59b31b0231ec1bc386396a2522d5c74a16540ad3245728f8f486e9024dadf42b"}
Aug 04 20:49:35.276992 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:35.276900    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:35.278040 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:35.278022    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-jfjfl" event={"ID":"03a04462-0175-400e-9968-f5f71987f040","Type":"ContainerStarted","Data":"ef09ccf49cca088019663291f6171a7d5734fd5ed7e1f8582b1fe9d826a444da"}
Aug 04 20:49:35.316656 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:35.316608    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m" podStartSLOduration=1.299779956 podStartE2EDuration="6.316591014s" totalImagesPullingTime="5.016811058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:49:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:49:35.314959054 +0000 UTC m=+522.558578337" watchObservedRunningTime="2026-08-04 20:49:35.316591014 +0000 UTC m=+522.560210296"
Aug 04 20:49:35.339639 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:35.339553    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv" podStartSLOduration=1.421594863 podStartE2EDuration="4.339532669s" totalImagesPullingTime="2.917937806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:49:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:49:35.33750883 +0000 UTC m=+522.581128112" watchObservedRunningTime="2026-08-04 20:49:35.339532669 +0000 UTC m=+522.583151951"
Aug 04 20:49:37.289530 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:37.289490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-jfjfl" event={"ID":"03a04462-0175-400e-9968-f5f71987f040","Type":"ContainerStarted","Data":"e7dcaeeded1ddd526bb9e22506167a4cf833a29d5f681c7b86150cc399ce4cdf"}
Aug 04 20:49:37.289957 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:37.289647    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-jfjfl"
Aug 04 20:49:37.310230 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:37.310182    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-jfjfl" podStartSLOduration=1.279223178 podStartE2EDuration="3.310166396s" totalImagesPullingTime="2.030943218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:49:37.30826616 +0000 UTC m=+524.551885452" watchObservedRunningTime="2026-08-04 20:49:37.310166396 +0000 UTC m=+524.553785686"
Aug 04 20:49:39.242812 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:39.242788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-w2lr5"
Aug 04 20:49:46.284509 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:46.284437    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:46.284931 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:46.284511    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:48.014823 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.014785    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"]
Aug 04 20:49:48.019758 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.019741    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.028426 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.028392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/076bfb06-5200-445d-ac56-cedf1dd7132a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-pdq7g\" (UID: \"076bfb06-5200-445d-ac56-cedf1dd7132a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.028532 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.028467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmnr5\" (UniqueName: \"kubernetes.io/projected/076bfb06-5200-445d-ac56-cedf1dd7132a-kube-api-access-gmnr5\") pod \"kuadrant-operator-controller-manager-6bd956ccf-pdq7g\" (UID: \"076bfb06-5200-445d-ac56-cedf1dd7132a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.029970 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.029948    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"]
Aug 04 20:49:48.115702 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.115666    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"]
Aug 04 20:49:48.115905 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.115882    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" containerName="manager" containerID="cri-o://0f6b36a2beb71015b2f54e042c2aa68f62c0b25f663d90c930bf2807816801d1" gracePeriod=2
Aug 04 20:49:48.128456 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.128431    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"]
Aug 04 20:49:48.128971 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.128945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/076bfb06-5200-445d-ac56-cedf1dd7132a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-pdq7g\" (UID: \"076bfb06-5200-445d-ac56-cedf1dd7132a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.129101 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.129037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmnr5\" (UniqueName: \"kubernetes.io/projected/076bfb06-5200-445d-ac56-cedf1dd7132a-kube-api-access-gmnr5\") pod \"kuadrant-operator-controller-manager-6bd956ccf-pdq7g\" (UID: \"076bfb06-5200-445d-ac56-cedf1dd7132a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.129628 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.129553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/076bfb06-5200-445d-ac56-cedf1dd7132a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-pdq7g\" (UID: \"076bfb06-5200-445d-ac56-cedf1dd7132a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.132070 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.132044    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"2a2c4492-8eca-45bc-a03c-f04d0c4631e1\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-4rl8m\": pods \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:48.136086 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.136066    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"]
Aug 04 20:49:48.136294 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.136274    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-gmnr5], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g" podUID="076bfb06-5200-445d-ac56-cedf1dd7132a"
Aug 04 20:49:48.149074 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.149050    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"]
Aug 04 20:49:48.153735 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.153715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmnr5\" (UniqueName: \"kubernetes.io/projected/076bfb06-5200-445d-ac56-cedf1dd7132a-kube-api-access-gmnr5\") pod \"kuadrant-operator-controller-manager-6bd956ccf-pdq7g\" (UID: \"076bfb06-5200-445d-ac56-cedf1dd7132a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.169662 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.169639    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"]
Aug 04 20:49:48.180014 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.179988    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"]
Aug 04 20:49:48.180503 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.180486    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" containerName="manager"
Aug 04 20:49:48.180575 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.180507    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" containerName="manager"
Aug 04 20:49:48.180623 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.180610    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" containerName="manager"
Aug 04 20:49:48.184056 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.184038    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 20:49:48.186998 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.186963    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:48.189674 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.189656    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"]
Aug 04 20:49:48.189873 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.189851    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" containerName="manager" containerID="cri-o://59b31b0231ec1bc386396a2522d5c74a16540ad3245728f8f486e9024dadf42b" gracePeriod=2
Aug 04 20:49:48.201599 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.201581    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"]
Aug 04 20:49:48.204039 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.204016    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"]
Aug 04 20:49:48.219041 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.218859    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:48.221098 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.220770    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qcgvv\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:48.221855 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.221834    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb"]
Aug 04 20:49:48.222291 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.222274    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" containerName="manager"
Aug 04 20:49:48.222291 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.222292    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" containerName="manager"
Aug 04 20:49:48.222442 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.222403    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" containerName="manager"
Aug 04 20:49:48.225682 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.225662    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb"
Aug 04 20:49:48.230436 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.230414    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cxnt\" (UniqueName: \"kubernetes.io/projected/a8caded2-8d9a-4c3f-a37f-93295d41ef4f-kube-api-access-8cxnt\") pod \"limitador-operator-controller-manager-6c549b6d85-r52cb\" (UID: \"a8caded2-8d9a-4c3f-a37f-93295d41ef4f\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb"
Aug 04 20:49:48.230538 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.230439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh5jw\" (UniqueName: \"kubernetes.io/projected/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-kube-api-access-dh5jw\") pod \"kuadrant-operator-controller-manager-68cf589589-cmsrx\" (UID: \"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 20:49:48.230538 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.230488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-cmsrx\" (UID: \"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 20:49:48.236059 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.236042    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb"]
Aug 04 20:49:48.255345 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.255283    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qcgvv\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:48.257442 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.257410    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:48.296844 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.296772    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-jfjfl"
Aug 04 20:49:48.331053 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.331022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cxnt\" (UniqueName: \"kubernetes.io/projected/a8caded2-8d9a-4c3f-a37f-93295d41ef4f-kube-api-access-8cxnt\") pod \"limitador-operator-controller-manager-6c549b6d85-r52cb\" (UID: \"a8caded2-8d9a-4c3f-a37f-93295d41ef4f\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb"
Aug 04 20:49:48.331053 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.331056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh5jw\" (UniqueName: \"kubernetes.io/projected/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-kube-api-access-dh5jw\") pod \"kuadrant-operator-controller-manager-68cf589589-cmsrx\" (UID: \"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 20:49:48.331338 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.331103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-cmsrx\" (UID: \"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 20:49:48.331684 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.331542    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-cmsrx\" (UID: \"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 20:49:48.332472 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.332443    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:48.335183 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.335158    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qcgvv\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:48.344743 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.344717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cxnt\" (UniqueName: \"kubernetes.io/projected/a8caded2-8d9a-4c3f-a37f-93295d41ef4f-kube-api-access-8cxnt\") pod \"limitador-operator-controller-manager-6c549b6d85-r52cb\" (UID: \"a8caded2-8d9a-4c3f-a37f-93295d41ef4f\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb"
Aug 04 20:49:48.347862 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.347841    2563 generic.go:356] "Generic (PLEG): container finished" podID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" containerID="0f6b36a2beb71015b2f54e042c2aa68f62c0b25f663d90c930bf2807816801d1" exitCode=0
Aug 04 20:49:48.349612 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.349591    2563 generic.go:356] "Generic (PLEG): container finished" podID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" containerID="59b31b0231ec1bc386396a2522d5c74a16540ad3245728f8f486e9024dadf42b" exitCode=0
Aug 04 20:49:48.349694 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.349644    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.350679 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.350657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh5jw\" (UniqueName: \"kubernetes.io/projected/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-kube-api-access-dh5jw\") pod \"kuadrant-operator-controller-manager-68cf589589-cmsrx\" (UID: \"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 20:49:48.352248 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.352223    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:48.354427 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.354393    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-pdq7g\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="076bfb06-5200-445d-ac56-cedf1dd7132a" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.356568 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.356527    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qcgvv\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:48.365176 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.365149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.367649 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.367624    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-pdq7g\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="076bfb06-5200-445d-ac56-cedf1dd7132a" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.370236 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.370213    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qcgvv\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:48.372465 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.372425    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:48.431864 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.431842    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmnr5\" (UniqueName: \"kubernetes.io/projected/076bfb06-5200-445d-ac56-cedf1dd7132a-kube-api-access-gmnr5\") pod \"076bfb06-5200-445d-ac56-cedf1dd7132a\" (UID: \"076bfb06-5200-445d-ac56-cedf1dd7132a\") "
Aug 04 20:49:48.431988 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.431883    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/076bfb06-5200-445d-ac56-cedf1dd7132a-extensions-socket-volume\") pod \"076bfb06-5200-445d-ac56-cedf1dd7132a\" (UID: \"076bfb06-5200-445d-ac56-cedf1dd7132a\") "
Aug 04 20:49:48.432159 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.432133    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076bfb06-5200-445d-ac56-cedf1dd7132a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "076bfb06-5200-445d-ac56-cedf1dd7132a" (UID: "076bfb06-5200-445d-ac56-cedf1dd7132a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:49:48.434141 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.434114    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/076bfb06-5200-445d-ac56-cedf1dd7132a-kube-api-access-gmnr5" (OuterVolumeSpecName: "kube-api-access-gmnr5") pod "076bfb06-5200-445d-ac56-cedf1dd7132a" (UID: "076bfb06-5200-445d-ac56-cedf1dd7132a"). InnerVolumeSpecName "kube-api-access-gmnr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:49:48.438844 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.438823    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:48.441220 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.441196    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-pdq7g\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="076bfb06-5200-445d-ac56-cedf1dd7132a" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.443398 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.443368    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qcgvv\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:48.443674 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.443660    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:48.445646 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.445623    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:48.448359 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.448296    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-4rl8m\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:48.451733 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.451714    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-pdq7g\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="076bfb06-5200-445d-ac56-cedf1dd7132a" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:48.457521 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:48.457503    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-qcgvv\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:48.496429 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.496403    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 20:49:48.533186 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.533125    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76q5l\" (UniqueName: \"kubernetes.io/projected/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-kube-api-access-76q5l\") pod \"2a2c4492-8eca-45bc-a03c-f04d0c4631e1\" (UID: \"2a2c4492-8eca-45bc-a03c-f04d0c4631e1\") "
Aug 04 20:49:48.533343 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.533190    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6h8k7\" (UniqueName: \"kubernetes.io/projected/5552316b-cfc8-4c29-b82b-03ffe1fa6e91-kube-api-access-6h8k7\") pod \"5552316b-cfc8-4c29-b82b-03ffe1fa6e91\" (UID: \"5552316b-cfc8-4c29-b82b-03ffe1fa6e91\") "
Aug 04 20:49:48.533343 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.533230    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-extensions-socket-volume\") pod \"2a2c4492-8eca-45bc-a03c-f04d0c4631e1\" (UID: \"2a2c4492-8eca-45bc-a03c-f04d0c4631e1\") "
Aug 04 20:49:48.534019 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.533485    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmnr5\" (UniqueName: \"kubernetes.io/projected/076bfb06-5200-445d-ac56-cedf1dd7132a-kube-api-access-gmnr5\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:48.534019 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.533507    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/076bfb06-5200-445d-ac56-cedf1dd7132a-extensions-socket-volume\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:48.534019 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.533979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2a2c4492-8eca-45bc-a03c-f04d0c4631e1" (UID: "2a2c4492-8eca-45bc-a03c-f04d0c4631e1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 20:49:48.536778 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.536750    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-kube-api-access-76q5l" (OuterVolumeSpecName: "kube-api-access-76q5l") pod "2a2c4492-8eca-45bc-a03c-f04d0c4631e1" (UID: "2a2c4492-8eca-45bc-a03c-f04d0c4631e1"). InnerVolumeSpecName "kube-api-access-76q5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:49:48.536778 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.536758    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5552316b-cfc8-4c29-b82b-03ffe1fa6e91-kube-api-access-6h8k7" (OuterVolumeSpecName: "kube-api-access-6h8k7") pod "5552316b-cfc8-4c29-b82b-03ffe1fa6e91" (UID: "5552316b-cfc8-4c29-b82b-03ffe1fa6e91"). InnerVolumeSpecName "kube-api-access-6h8k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:49:48.573105 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.573073    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb"
Aug 04 20:49:48.634155 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.634066    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76q5l\" (UniqueName: \"kubernetes.io/projected/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-kube-api-access-76q5l\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:48.634155 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.634098    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6h8k7\" (UniqueName: \"kubernetes.io/projected/5552316b-cfc8-4c29-b82b-03ffe1fa6e91-kube-api-access-6h8k7\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:48.634155 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.634111    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a2c4492-8eca-45bc-a03c-f04d0c4631e1-extensions-socket-volume\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:49:48.636653 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.636620    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"]
Aug 04 20:49:48.638506 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:49:48.638483    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ad76b9_d8e9_4fb3_835d_8554e7325ea0.slice/crio-26a8555ab65e75c3e02964f072ad170770de6c32a182ea181508056547dd7a43 WatchSource:0}: Error finding container 26a8555ab65e75c3e02964f072ad170770de6c32a182ea181508056547dd7a43: Status 404 returned error can't find the container with id 26a8555ab65e75c3e02964f072ad170770de6c32a182ea181508056547dd7a43
Aug 04 20:49:48.722690 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:48.722663    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb"]
Aug 04 20:49:48.724739 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:49:48.724715    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8caded2_8d9a_4c3f_a37f_93295d41ef4f.slice/crio-7a7b82dcb24f18a7e032f4ff015091d506b49f3ca4f7abb88bf113ce3e6dd19f WatchSource:0}: Error finding container 7a7b82dcb24f18a7e032f4ff015091d506b49f3ca4f7abb88bf113ce3e6dd19f: Status 404 returned error can't find the container with id 7a7b82dcb24f18a7e032f4ff015091d506b49f3ca4f7abb88bf113ce3e6dd19f
Aug 04 20:49:49.243279 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.243250    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="076bfb06-5200-445d-ac56-cedf1dd7132a" path="/var/lib/kubelet/pods/076bfb06-5200-445d-ac56-cedf1dd7132a/volumes"
Aug 04 20:49:49.243634 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.243520    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a2c4492-8eca-45bc-a03c-f04d0c4631e1" path="/var/lib/kubelet/pods/2a2c4492-8eca-45bc-a03c-f04d0c4631e1/volumes"
Aug 04 20:49:49.243784 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.243772    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5552316b-cfc8-4c29-b82b-03ffe1fa6e91" path="/var/lib/kubelet/pods/5552316b-cfc8-4c29-b82b-03ffe1fa6e91/volumes"
Aug 04 20:49:49.356190 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.356163    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-qcgvv"
Aug 04 20:49:49.356356 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.356191    2563 scope.go:117] "RemoveContainer" containerID="59b31b0231ec1bc386396a2522d5c74a16540ad3245728f8f486e9024dadf42b"
Aug 04 20:49:49.358184 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.358134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb" event={"ID":"a8caded2-8d9a-4c3f-a37f-93295d41ef4f","Type":"ContainerStarted","Data":"98afd59c59e32a5a4faea241631404932aa6a4ff75f659c8c121e8afc7fb1ee5"}
Aug 04 20:49:49.358302 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.358235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb" event={"ID":"a8caded2-8d9a-4c3f-a37f-93295d41ef4f","Type":"ContainerStarted","Data":"7a7b82dcb24f18a7e032f4ff015091d506b49f3ca4f7abb88bf113ce3e6dd19f"}
Aug 04 20:49:49.358302 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.358282    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb"
Aug 04 20:49:49.359932 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.359910    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx" event={"ID":"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0","Type":"ContainerStarted","Data":"99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353"}
Aug 04 20:49:49.360037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.359946    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx" event={"ID":"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0","Type":"ContainerStarted","Data":"26a8555ab65e75c3e02964f072ad170770de6c32a182ea181508056547dd7a43"}
Aug 04 20:49:49.360037 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.359992    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 20:49:49.361267 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.361252    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:49.361398 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.361254    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-4rl8m"
Aug 04 20:49:49.365796 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.365779    2563 scope.go:117] "RemoveContainer" containerID="0f6b36a2beb71015b2f54e042c2aa68f62c0b25f663d90c930bf2807816801d1"
Aug 04 20:49:49.402347 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.402244    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb" podStartSLOduration=1.402231249 podStartE2EDuration="1.402231249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:49:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:49:49.401191768 +0000 UTC m=+536.644811049" watchObservedRunningTime="2026-08-04 20:49:49.402231249 +0000 UTC m=+536.645850530"
Aug 04 20:49:49.404261 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:49:49.404239    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-pdq7g\" is forbidden: User \"system:node:ip-10-0-129-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-249.ec2.internal' and this object" podUID="076bfb06-5200-445d-ac56-cedf1dd7132a" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-pdq7g"
Aug 04 20:49:49.456632 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:49:49.456596    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx" podStartSLOduration=1.456584437 podStartE2EDuration="1.456584437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:49:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:49:49.452305771 +0000 UTC m=+536.695925053" watchObservedRunningTime="2026-08-04 20:49:49.456584437 +0000 UTC m=+536.700203718"
Aug 04 20:50:00.371068 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:00.371039    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 20:50:00.371510 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:00.371103    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-r52cb"
Aug 04 20:50:18.671450 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.671418    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"]
Aug 04 20:50:18.673784 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.673768    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:18.676473 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.676452    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 04 20:50:18.676718 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.676700    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qht4l\""
Aug 04 20:50:18.683055 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.683033    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"]
Aug 04 20:50:18.770198 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.770160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtsvl\" (UniqueName: \"kubernetes.io/projected/a9fa9750-b708-492c-8fa8-3f149f50617f-kube-api-access-xtsvl\") pod \"limitador-limitador-859c4d96d6-x6n8r\" (UID: \"a9fa9750-b708-492c-8fa8-3f149f50617f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:18.770557 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.770537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a9fa9750-b708-492c-8fa8-3f149f50617f-config-file\") pod \"limitador-limitador-859c4d96d6-x6n8r\" (UID: \"a9fa9750-b708-492c-8fa8-3f149f50617f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:18.773358 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.773332    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"]
Aug 04 20:50:18.872050 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.872017    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a9fa9750-b708-492c-8fa8-3f149f50617f-config-file\") pod \"limitador-limitador-859c4d96d6-x6n8r\" (UID: \"a9fa9750-b708-492c-8fa8-3f149f50617f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:18.872231 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.872062    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtsvl\" (UniqueName: \"kubernetes.io/projected/a9fa9750-b708-492c-8fa8-3f149f50617f-kube-api-access-xtsvl\") pod \"limitador-limitador-859c4d96d6-x6n8r\" (UID: \"a9fa9750-b708-492c-8fa8-3f149f50617f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:18.872902 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.872869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a9fa9750-b708-492c-8fa8-3f149f50617f-config-file\") pod \"limitador-limitador-859c4d96d6-x6n8r\" (UID: \"a9fa9750-b708-492c-8fa8-3f149f50617f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:18.881082 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.881054    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtsvl\" (UniqueName: \"kubernetes.io/projected/a9fa9750-b708-492c-8fa8-3f149f50617f-kube-api-access-xtsvl\") pod \"limitador-limitador-859c4d96d6-x6n8r\" (UID: \"a9fa9750-b708-492c-8fa8-3f149f50617f\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:18.984887 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:18.984815    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:19.121363 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.121335    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"]
Aug 04 20:50:19.123127 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:50:19.123096    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9fa9750_b708_492c_8fa8_3f149f50617f.slice/crio-d99a3c31db55bb1ed535f318bcdcbe44f807edcb091010273e965bfd0df0144c WatchSource:0}: Error finding container d99a3c31db55bb1ed535f318bcdcbe44f807edcb091010273e965bfd0df0144c: Status 404 returned error can't find the container with id d99a3c31db55bb1ed535f318bcdcbe44f807edcb091010273e965bfd0df0144c
Aug 04 20:50:19.470194 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.470165    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-qkv2c"]
Aug 04 20:50:19.473526 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.473506    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-qkv2c"
Aug 04 20:50:19.475758 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.475737    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2fqpx\""
Aug 04 20:50:19.480991 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.480969    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-qkv2c"]
Aug 04 20:50:19.506529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.506494    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r" event={"ID":"a9fa9750-b708-492c-8fa8-3f149f50617f","Type":"ContainerStarted","Data":"d99a3c31db55bb1ed535f318bcdcbe44f807edcb091010273e965bfd0df0144c"}
Aug 04 20:50:19.577472 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.577439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z8np\" (UniqueName: \"kubernetes.io/projected/53d28bfe-75fc-4f44-a1ae-7607320a6e5b-kube-api-access-4z8np\") pod \"authorino-b54df94b4-qkv2c\" (UID: \"53d28bfe-75fc-4f44-a1ae-7607320a6e5b\") " pod="kuadrant-system/authorino-b54df94b4-qkv2c"
Aug 04 20:50:19.594294 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.594258    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-vcm4s"]
Aug 04 20:50:19.596722 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.596701    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-vcm4s"
Aug 04 20:50:19.604311 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.604286    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-vcm4s"]
Aug 04 20:50:19.678115 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.678088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z8np\" (UniqueName: \"kubernetes.io/projected/53d28bfe-75fc-4f44-a1ae-7607320a6e5b-kube-api-access-4z8np\") pod \"authorino-b54df94b4-qkv2c\" (UID: \"53d28bfe-75fc-4f44-a1ae-7607320a6e5b\") " pod="kuadrant-system/authorino-b54df94b4-qkv2c"
Aug 04 20:50:19.678496 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.678139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln646\" (UniqueName: \"kubernetes.io/projected/c8c87e14-0506-40ac-818f-ade15a9ac44c-kube-api-access-ln646\") pod \"authorino-c548b5d7d-vcm4s\" (UID: \"c8c87e14-0506-40ac-818f-ade15a9ac44c\") " pod="kuadrant-system/authorino-c548b5d7d-vcm4s"
Aug 04 20:50:19.686535 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.686516    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z8np\" (UniqueName: \"kubernetes.io/projected/53d28bfe-75fc-4f44-a1ae-7607320a6e5b-kube-api-access-4z8np\") pod \"authorino-b54df94b4-qkv2c\" (UID: \"53d28bfe-75fc-4f44-a1ae-7607320a6e5b\") " pod="kuadrant-system/authorino-b54df94b4-qkv2c"
Aug 04 20:50:19.779416 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.779331    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln646\" (UniqueName: \"kubernetes.io/projected/c8c87e14-0506-40ac-818f-ade15a9ac44c-kube-api-access-ln646\") pod \"authorino-c548b5d7d-vcm4s\" (UID: \"c8c87e14-0506-40ac-818f-ade15a9ac44c\") " pod="kuadrant-system/authorino-c548b5d7d-vcm4s"
Aug 04 20:50:19.782895 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.782870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-qkv2c"
Aug 04 20:50:19.788709 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.788690    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln646\" (UniqueName: \"kubernetes.io/projected/c8c87e14-0506-40ac-818f-ade15a9ac44c-kube-api-access-ln646\") pod \"authorino-c548b5d7d-vcm4s\" (UID: \"c8c87e14-0506-40ac-818f-ade15a9ac44c\") " pod="kuadrant-system/authorino-c548b5d7d-vcm4s"
Aug 04 20:50:19.905861 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.905821    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-vcm4s"
Aug 04 20:50:19.908827 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:19.908807    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-qkv2c"]
Aug 04 20:50:19.910296 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:50:19.910270    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53d28bfe_75fc_4f44_a1ae_7607320a6e5b.slice/crio-90d9e6ccd173a8b2fc1e29590a17e9d5c14f6e42340eb530b9809a11ca277756 WatchSource:0}: Error finding container 90d9e6ccd173a8b2fc1e29590a17e9d5c14f6e42340eb530b9809a11ca277756: Status 404 returned error can't find the container with id 90d9e6ccd173a8b2fc1e29590a17e9d5c14f6e42340eb530b9809a11ca277756
Aug 04 20:50:20.033558 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:20.033537    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-vcm4s"]
Aug 04 20:50:20.034873 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:50:20.034844    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8c87e14_0506_40ac_818f_ade15a9ac44c.slice/crio-5cce264340460cc1df03715f90a194948e5a1c7ad715f443ffcd69e806a242dc WatchSource:0}: Error finding container 5cce264340460cc1df03715f90a194948e5a1c7ad715f443ffcd69e806a242dc: Status 404 returned error can't find the container with id 5cce264340460cc1df03715f90a194948e5a1c7ad715f443ffcd69e806a242dc
Aug 04 20:50:20.512893 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:20.512855    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-vcm4s" event={"ID":"c8c87e14-0506-40ac-818f-ade15a9ac44c","Type":"ContainerStarted","Data":"5cce264340460cc1df03715f90a194948e5a1c7ad715f443ffcd69e806a242dc"}
Aug 04 20:50:20.514209 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:20.514176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-qkv2c" event={"ID":"53d28bfe-75fc-4f44-a1ae-7607320a6e5b","Type":"ContainerStarted","Data":"90d9e6ccd173a8b2fc1e29590a17e9d5c14f6e42340eb530b9809a11ca277756"}
Aug 04 20:50:21.525068 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:21.525031    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r" event={"ID":"a9fa9750-b708-492c-8fa8-3f149f50617f","Type":"ContainerStarted","Data":"ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5"}
Aug 04 20:50:21.525811 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:21.525790    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:21.545747 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:21.544851    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r" podStartSLOduration=2.039031174 podStartE2EDuration="3.544833737s" totalImagesPullingTime="1.505802563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:50:21.543844119 +0000 UTC m=+568.787463395" watchObservedRunningTime="2026-08-04 20:50:21.544833737 +0000 UTC m=+568.788453018"
Aug 04 20:50:22.530829 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:22.530789    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-vcm4s" event={"ID":"c8c87e14-0506-40ac-818f-ade15a9ac44c","Type":"ContainerStarted","Data":"c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291"}
Aug 04 20:50:22.532274 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:22.532251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-qkv2c" event={"ID":"53d28bfe-75fc-4f44-a1ae-7607320a6e5b","Type":"ContainerStarted","Data":"a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417"}
Aug 04 20:50:22.547613 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:22.547577    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-vcm4s" podStartSLOduration=1.541306594 podStartE2EDuration="3.547563032s" totalImagesPullingTime="2.006256438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:50:22.546886257 +0000 UTC m=+569.790505539" watchObservedRunningTime="2026-08-04 20:50:22.547563032 +0000 UTC m=+569.791182314"
Aug 04 20:50:22.561715 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:22.561676    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-qkv2c" podStartSLOduration=1.4214809019999999 podStartE2EDuration="3.561662551s" totalImagesPullingTime="2.140181649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:50:22.560085701 +0000 UTC m=+569.803704984" watchObservedRunningTime="2026-08-04 20:50:22.561662551 +0000 UTC m=+569.805281870"
Aug 04 20:50:22.585095 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:22.585066    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-qkv2c"]
Aug 04 20:50:24.542501 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:24.542448    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-qkv2c" podUID="53d28bfe-75fc-4f44-a1ae-7607320a6e5b" containerName="authorino" containerID="cri-o://a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417" gracePeriod=30
Aug 04 20:50:24.784678 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:24.784654    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-qkv2c"
Aug 04 20:50:24.924063 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:24.924032    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z8np\" (UniqueName: \"kubernetes.io/projected/53d28bfe-75fc-4f44-a1ae-7607320a6e5b-kube-api-access-4z8np\") pod \"53d28bfe-75fc-4f44-a1ae-7607320a6e5b\" (UID: \"53d28bfe-75fc-4f44-a1ae-7607320a6e5b\") "
Aug 04 20:50:24.926100 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:24.926072    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53d28bfe-75fc-4f44-a1ae-7607320a6e5b-kube-api-access-4z8np" (OuterVolumeSpecName: "kube-api-access-4z8np") pod "53d28bfe-75fc-4f44-a1ae-7607320a6e5b" (UID: "53d28bfe-75fc-4f44-a1ae-7607320a6e5b"). InnerVolumeSpecName "kube-api-access-4z8np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:50:25.024764 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:25.024733    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z8np\" (UniqueName: \"kubernetes.io/projected/53d28bfe-75fc-4f44-a1ae-7607320a6e5b-kube-api-access-4z8np\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:50:25.548823 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:25.548734    2563 generic.go:356] "Generic (PLEG): container finished" podID="53d28bfe-75fc-4f44-a1ae-7607320a6e5b" containerID="a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417" exitCode=0
Aug 04 20:50:25.548823 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:25.548778    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-qkv2c"
Aug 04 20:50:25.549235 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:25.548825    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-qkv2c" event={"ID":"53d28bfe-75fc-4f44-a1ae-7607320a6e5b","Type":"ContainerDied","Data":"a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417"}
Aug 04 20:50:25.549235 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:25.548867    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-qkv2c" event={"ID":"53d28bfe-75fc-4f44-a1ae-7607320a6e5b","Type":"ContainerDied","Data":"90d9e6ccd173a8b2fc1e29590a17e9d5c14f6e42340eb530b9809a11ca277756"}
Aug 04 20:50:25.549235 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:25.548883    2563 scope.go:117] "RemoveContainer" containerID="a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417"
Aug 04 20:50:25.557376 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:25.557336    2563 scope.go:117] "RemoveContainer" containerID="a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417"
Aug 04 20:50:25.557589 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:50:25.557571    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417\": container with ID starting with a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417 not found: ID does not exist" containerID="a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417"
Aug 04 20:50:25.557642 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:25.557597    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417"} err="failed to get container status \"a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417\": rpc error: code = NotFound desc = could not find container \"a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417\": container with ID starting with a068a2da2324d423d98a109b58179746d6c893eb6c1d87da48f5f266b8eec417 not found: ID does not exist"
Aug 04 20:50:25.566185 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:25.566163    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-qkv2c"]
Aug 04 20:50:25.572495 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:25.572465    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-qkv2c"]
Aug 04 20:50:27.245148 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:27.245109    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53d28bfe-75fc-4f44-a1ae-7607320a6e5b" path="/var/lib/kubelet/pods/53d28bfe-75fc-4f44-a1ae-7607320a6e5b/volumes"
Aug 04 20:50:32.460782 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:32.460749    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"]
Aug 04 20:50:32.461886 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:32.460937    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r" podUID="a9fa9750-b708-492c-8fa8-3f149f50617f" containerName="limitador" containerID="cri-o://ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5" gracePeriod=30
Aug 04 20:50:32.461886 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:32.461802    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:33.002202 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.002181    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:33.092259 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.092186    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a9fa9750-b708-492c-8fa8-3f149f50617f-config-file\") pod \"a9fa9750-b708-492c-8fa8-3f149f50617f\" (UID: \"a9fa9750-b708-492c-8fa8-3f149f50617f\") "
Aug 04 20:50:33.092259 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.092240    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtsvl\" (UniqueName: \"kubernetes.io/projected/a9fa9750-b708-492c-8fa8-3f149f50617f-kube-api-access-xtsvl\") pod \"a9fa9750-b708-492c-8fa8-3f149f50617f\" (UID: \"a9fa9750-b708-492c-8fa8-3f149f50617f\") "
Aug 04 20:50:33.092569 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.092533    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9fa9750-b708-492c-8fa8-3f149f50617f-config-file" (OuterVolumeSpecName: "config-file") pod "a9fa9750-b708-492c-8fa8-3f149f50617f" (UID: "a9fa9750-b708-492c-8fa8-3f149f50617f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 20:50:33.094377 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.094356    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9fa9750-b708-492c-8fa8-3f149f50617f-kube-api-access-xtsvl" (OuterVolumeSpecName: "kube-api-access-xtsvl") pod "a9fa9750-b708-492c-8fa8-3f149f50617f" (UID: "a9fa9750-b708-492c-8fa8-3f149f50617f"). InnerVolumeSpecName "kube-api-access-xtsvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:50:33.193696 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.193659    2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a9fa9750-b708-492c-8fa8-3f149f50617f-config-file\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:50:33.193696 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.193694    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtsvl\" (UniqueName: \"kubernetes.io/projected/a9fa9750-b708-492c-8fa8-3f149f50617f-kube-api-access-xtsvl\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:50:33.596059 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.596023    2563 generic.go:356] "Generic (PLEG): container finished" podID="a9fa9750-b708-492c-8fa8-3f149f50617f" containerID="ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5" exitCode=0
Aug 04 20:50:33.596581 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.596072    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"
Aug 04 20:50:33.596581 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.596091    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r" event={"ID":"a9fa9750-b708-492c-8fa8-3f149f50617f","Type":"ContainerDied","Data":"ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5"}
Aug 04 20:50:33.596581 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.596159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-x6n8r" event={"ID":"a9fa9750-b708-492c-8fa8-3f149f50617f","Type":"ContainerDied","Data":"d99a3c31db55bb1ed535f318bcdcbe44f807edcb091010273e965bfd0df0144c"}
Aug 04 20:50:33.596581 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.596184    2563 scope.go:117] "RemoveContainer" containerID="ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5"
Aug 04 20:50:33.605029 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.605011    2563 scope.go:117] "RemoveContainer" containerID="ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5"
Aug 04 20:50:33.605279 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:50:33.605263    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5\": container with ID starting with ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5 not found: ID does not exist" containerID="ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5"
Aug 04 20:50:33.605354 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.605286    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5"} err="failed to get container status \"ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5\": rpc error: code = NotFound desc = could not find container \"ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5\": container with ID starting with ad8d9c941ec86d3787aaef86d5f2990a88d77aace8b25b5dc5e15fe4a8482eb5 not found: ID does not exist"
Aug 04 20:50:33.613745 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.613723    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"]
Aug 04 20:50:33.619369 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:33.619349    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x6n8r"]
Aug 04 20:50:35.229427 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.229398    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"]
Aug 04 20:50:35.229903 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.229725    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d28bfe-75fc-4f44-a1ae-7607320a6e5b" containerName="authorino"
Aug 04 20:50:35.229903 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.229736    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d28bfe-75fc-4f44-a1ae-7607320a6e5b" containerName="authorino"
Aug 04 20:50:35.229903 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.229746    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9fa9750-b708-492c-8fa8-3f149f50617f" containerName="limitador"
Aug 04 20:50:35.229903 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.229751    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9fa9750-b708-492c-8fa8-3f149f50617f" containerName="limitador"
Aug 04 20:50:35.229903 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.229801    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9fa9750-b708-492c-8fa8-3f149f50617f" containerName="limitador"
Aug 04 20:50:35.229903 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.229807    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="53d28bfe-75fc-4f44-a1ae-7607320a6e5b" containerName="authorino"
Aug 04 20:50:35.232503 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.232483    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"
Aug 04 20:50:35.235293 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.235159    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Aug 04 20:50:35.235293 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.235247    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 20:50:35.236247 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.236210    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Aug 04 20:50:35.236247 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.236209    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zbrjg\""
Aug 04 20:50:35.245084 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.245061    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9fa9750-b708-492c-8fa8-3f149f50617f" path="/var/lib/kubelet/pods/a9fa9750-b708-492c-8fa8-3f149f50617f/volumes"
Aug 04 20:50:35.245310 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.245298    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"]
Aug 04 20:50:35.309197 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.309162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgzhx\" (UniqueName: \"kubernetes.io/projected/c2104bfe-57ee-4590-90c9-4ddda089ba85-kube-api-access-kgzhx\") pod \"postgres-69fb4594cf-phwdq\" (UID: \"c2104bfe-57ee-4590-90c9-4ddda089ba85\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"
Aug 04 20:50:35.309394 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.309219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c2104bfe-57ee-4590-90c9-4ddda089ba85-data\") pod \"postgres-69fb4594cf-phwdq\" (UID: \"c2104bfe-57ee-4590-90c9-4ddda089ba85\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"
Aug 04 20:50:35.410500 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.410468    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c2104bfe-57ee-4590-90c9-4ddda089ba85-data\") pod \"postgres-69fb4594cf-phwdq\" (UID: \"c2104bfe-57ee-4590-90c9-4ddda089ba85\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"
Aug 04 20:50:35.410699 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.410586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgzhx\" (UniqueName: \"kubernetes.io/projected/c2104bfe-57ee-4590-90c9-4ddda089ba85-kube-api-access-kgzhx\") pod \"postgres-69fb4594cf-phwdq\" (UID: \"c2104bfe-57ee-4590-90c9-4ddda089ba85\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"
Aug 04 20:50:35.410930 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.410909    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c2104bfe-57ee-4590-90c9-4ddda089ba85-data\") pod \"postgres-69fb4594cf-phwdq\" (UID: \"c2104bfe-57ee-4590-90c9-4ddda089ba85\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"
Aug 04 20:50:35.419048 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.419028    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgzhx\" (UniqueName: \"kubernetes.io/projected/c2104bfe-57ee-4590-90c9-4ddda089ba85-kube-api-access-kgzhx\") pod \"postgres-69fb4594cf-phwdq\" (UID: \"c2104bfe-57ee-4590-90c9-4ddda089ba85\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"
Aug 04 20:50:35.545672 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.545583    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"
Aug 04 20:50:35.676881 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:35.676854    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"]
Aug 04 20:50:35.678606 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:50:35.678579    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2104bfe_57ee_4590_90c9_4ddda089ba85.slice/crio-7a4e26192cd44f21d174434c48f28ff08bef124ae14a7fd3786ef8d7803ae1cb WatchSource:0}: Error finding container 7a4e26192cd44f21d174434c48f28ff08bef124ae14a7fd3786ef8d7803ae1cb: Status 404 returned error can't find the container with id 7a4e26192cd44f21d174434c48f28ff08bef124ae14a7fd3786ef8d7803ae1cb
Aug 04 20:50:36.618899 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:36.618850    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq" event={"ID":"c2104bfe-57ee-4590-90c9-4ddda089ba85","Type":"ContainerStarted","Data":"7a4e26192cd44f21d174434c48f28ff08bef124ae14a7fd3786ef8d7803ae1cb"}
Aug 04 20:50:40.645858 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:40.645826    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq" event={"ID":"c2104bfe-57ee-4590-90c9-4ddda089ba85","Type":"ContainerStarted","Data":"cc6c7a7ee6d918e630f0c27173c98eb42db284c36618e607b3c6c44e66d8d381"}
Aug 04 20:50:40.646258 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:40.645897    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"
Aug 04 20:50:40.663134 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:40.663089    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq" podStartSLOduration=0.991632378 podStartE2EDuration="5.6630765s" totalImagesPullingTime="4.671444122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:50:40.662189397 +0000 UTC m=+587.905808678" watchObservedRunningTime="2026-08-04 20:50:40.6630765 +0000 UTC m=+587.906695780"
Aug 04 20:50:46.680768 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:46.680735    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-phwdq"
Aug 04 20:50:47.276559 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.276526    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxhqf"]
Aug 04 20:50:47.279778 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.279757    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf"
Aug 04 20:50:47.288903 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.288882    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxhqf"]
Aug 04 20:50:47.315151 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.315122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrkdm\" (UniqueName: \"kubernetes.io/projected/f7085976-8975-4356-a3d1-8ca26ae44849-kube-api-access-wrkdm\") pod \"authorino-5dc7ccb5f-fxhqf\" (UID: \"f7085976-8975-4356-a3d1-8ca26ae44849\") " pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf"
Aug 04 20:50:47.415786 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.415748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrkdm\" (UniqueName: \"kubernetes.io/projected/f7085976-8975-4356-a3d1-8ca26ae44849-kube-api-access-wrkdm\") pod \"authorino-5dc7ccb5f-fxhqf\" (UID: \"f7085976-8975-4356-a3d1-8ca26ae44849\") " pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf"
Aug 04 20:50:47.425009 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.424982    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrkdm\" (UniqueName: \"kubernetes.io/projected/f7085976-8975-4356-a3d1-8ca26ae44849-kube-api-access-wrkdm\") pod \"authorino-5dc7ccb5f-fxhqf\" (UID: \"f7085976-8975-4356-a3d1-8ca26ae44849\") " pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf"
Aug 04 20:50:47.484275 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.484248    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxhqf"]
Aug 04 20:50:47.484529 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.484516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf"
Aug 04 20:50:47.511999 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.511969    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7455c65646-4prrg"]
Aug 04 20:50:47.515560 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.515541    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7455c65646-4prrg"
Aug 04 20:50:47.516184 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.516163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vz7v\" (UniqueName: \"kubernetes.io/projected/9d034b84-52a9-401f-be06-64fb74d279f8-kube-api-access-5vz7v\") pod \"authorino-7455c65646-4prrg\" (UID: \"9d034b84-52a9-401f-be06-64fb74d279f8\") " pod="kuadrant-system/authorino-7455c65646-4prrg"
Aug 04 20:50:47.529179 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.527356    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7455c65646-4prrg"]
Aug 04 20:50:47.601392 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.601339    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7455c65646-4prrg"]
Aug 04 20:50:47.601665 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:50:47.601640    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-5vz7v], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7455c65646-4prrg" podUID="9d034b84-52a9-401f-be06-64fb74d279f8"
Aug 04 20:50:47.617605 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.617575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vz7v\" (UniqueName: \"kubernetes.io/projected/9d034b84-52a9-401f-be06-64fb74d279f8-kube-api-access-5vz7v\") pod \"authorino-7455c65646-4prrg\" (UID: \"9d034b84-52a9-401f-be06-64fb74d279f8\") " pod="kuadrant-system/authorino-7455c65646-4prrg"
Aug 04 20:50:47.625849 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.625822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vz7v\" (UniqueName: \"kubernetes.io/projected/9d034b84-52a9-401f-be06-64fb74d279f8-kube-api-access-5vz7v\") pod \"authorino-7455c65646-4prrg\" (UID: \"9d034b84-52a9-401f-be06-64fb74d279f8\") " pod="kuadrant-system/authorino-7455c65646-4prrg"
Aug 04 20:50:47.629246 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.629225    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79d565c59b-24pvp"]
Aug 04 20:50:47.632474 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.632457    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79d565c59b-24pvp"
Aug 04 20:50:47.635190 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.635170    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Aug 04 20:50:47.640395 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.640377    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxhqf"]
Aug 04 20:50:47.642251 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:50:47.642229    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7085976_8975_4356_a3d1_8ca26ae44849.slice/crio-da03fa5f910ad64ac7ea1a1846a5c90cfdecfce541ddafb6c7168bdf5f63ed71 WatchSource:0}: Error finding container da03fa5f910ad64ac7ea1a1846a5c90cfdecfce541ddafb6c7168bdf5f63ed71: Status 404 returned error can't find the container with id da03fa5f910ad64ac7ea1a1846a5c90cfdecfce541ddafb6c7168bdf5f63ed71
Aug 04 20:50:47.650601 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.650578    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79d565c59b-24pvp"]
Aug 04 20:50:47.684185 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.684151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf" event={"ID":"f7085976-8975-4356-a3d1-8ca26ae44849","Type":"ContainerStarted","Data":"da03fa5f910ad64ac7ea1a1846a5c90cfdecfce541ddafb6c7168bdf5f63ed71"}
Aug 04 20:50:47.684572 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.684165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7455c65646-4prrg"
Aug 04 20:50:47.690946 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.690924    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7455c65646-4prrg"
Aug 04 20:50:47.718915 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.718883    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7vp9\" (UniqueName: \"kubernetes.io/projected/397210c0-6737-4bf3-84c7-ebfa58c5d355-kube-api-access-p7vp9\") pod \"authorino-79d565c59b-24pvp\" (UID: \"397210c0-6737-4bf3-84c7-ebfa58c5d355\") " pod="kuadrant-system/authorino-79d565c59b-24pvp"
Aug 04 20:50:47.719039 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.718919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/397210c0-6737-4bf3-84c7-ebfa58c5d355-tls-cert\") pod \"authorino-79d565c59b-24pvp\" (UID: \"397210c0-6737-4bf3-84c7-ebfa58c5d355\") " pod="kuadrant-system/authorino-79d565c59b-24pvp"
Aug 04 20:50:47.819363 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.819264    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vz7v\" (UniqueName: \"kubernetes.io/projected/9d034b84-52a9-401f-be06-64fb74d279f8-kube-api-access-5vz7v\") pod \"9d034b84-52a9-401f-be06-64fb74d279f8\" (UID: \"9d034b84-52a9-401f-be06-64fb74d279f8\") "
Aug 04 20:50:47.819485 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.819419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7vp9\" (UniqueName: \"kubernetes.io/projected/397210c0-6737-4bf3-84c7-ebfa58c5d355-kube-api-access-p7vp9\") pod \"authorino-79d565c59b-24pvp\" (UID: \"397210c0-6737-4bf3-84c7-ebfa58c5d355\") " pod="kuadrant-system/authorino-79d565c59b-24pvp"
Aug 04 20:50:47.819485 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.819442    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/397210c0-6737-4bf3-84c7-ebfa58c5d355-tls-cert\") pod \"authorino-79d565c59b-24pvp\" (UID: \"397210c0-6737-4bf3-84c7-ebfa58c5d355\") " pod="kuadrant-system/authorino-79d565c59b-24pvp"
Aug 04 20:50:47.821437 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.821403    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d034b84-52a9-401f-be06-64fb74d279f8-kube-api-access-5vz7v" (OuterVolumeSpecName: "kube-api-access-5vz7v") pod "9d034b84-52a9-401f-be06-64fb74d279f8" (UID: "9d034b84-52a9-401f-be06-64fb74d279f8"). InnerVolumeSpecName "kube-api-access-5vz7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:50:47.821878 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.821862    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/397210c0-6737-4bf3-84c7-ebfa58c5d355-tls-cert\") pod \"authorino-79d565c59b-24pvp\" (UID: \"397210c0-6737-4bf3-84c7-ebfa58c5d355\") " pod="kuadrant-system/authorino-79d565c59b-24pvp"
Aug 04 20:50:47.828637 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.828619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7vp9\" (UniqueName: \"kubernetes.io/projected/397210c0-6737-4bf3-84c7-ebfa58c5d355-kube-api-access-p7vp9\") pod \"authorino-79d565c59b-24pvp\" (UID: \"397210c0-6737-4bf3-84c7-ebfa58c5d355\") " pod="kuadrant-system/authorino-79d565c59b-24pvp"
Aug 04 20:50:47.920110 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.920084    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vz7v\" (UniqueName: \"kubernetes.io/projected/9d034b84-52a9-401f-be06-64fb74d279f8-kube-api-access-5vz7v\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:50:47.943022 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:47.943000    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79d565c59b-24pvp"
Aug 04 20:50:48.073054 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.073028    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79d565c59b-24pvp"]
Aug 04 20:50:48.074570 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:50:48.074544    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod397210c0_6737_4bf3_84c7_ebfa58c5d355.slice/crio-eca987c52e85412e36d5f00d50b79bac1b81a671f7b85065f13c17b25c747bc9 WatchSource:0}: Error finding container eca987c52e85412e36d5f00d50b79bac1b81a671f7b85065f13c17b25c747bc9: Status 404 returned error can't find the container with id eca987c52e85412e36d5f00d50b79bac1b81a671f7b85065f13c17b25c747bc9
Aug 04 20:50:48.691266 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.691181    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79d565c59b-24pvp" event={"ID":"397210c0-6737-4bf3-84c7-ebfa58c5d355","Type":"ContainerStarted","Data":"75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3"}
Aug 04 20:50:48.691266 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.691226    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79d565c59b-24pvp" event={"ID":"397210c0-6737-4bf3-84c7-ebfa58c5d355","Type":"ContainerStarted","Data":"eca987c52e85412e36d5f00d50b79bac1b81a671f7b85065f13c17b25c747bc9"}
Aug 04 20:50:48.692723 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.692704    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7455c65646-4prrg"
Aug 04 20:50:48.692846 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.692758    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf" podUID="f7085976-8975-4356-a3d1-8ca26ae44849" containerName="authorino" containerID="cri-o://567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd" gracePeriod=30
Aug 04 20:50:48.692846 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.692699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf" event={"ID":"f7085976-8975-4356-a3d1-8ca26ae44849","Type":"ContainerStarted","Data":"567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd"}
Aug 04 20:50:48.708735 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.708694    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-79d565c59b-24pvp" podStartSLOduration=1.218605633 podStartE2EDuration="1.708678009s" totalImagesPullingTime="490.072376ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:50:48.707643049 +0000 UTC m=+595.951262333" watchObservedRunningTime="2026-08-04 20:50:48.708678009 +0000 UTC m=+595.952297296"
Aug 04 20:50:48.723299 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.723250    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf" podStartSLOduration=1.10833079 podStartE2EDuration="1.723233933s" totalImagesPullingTime="614.903143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:50:48.722879455 +0000 UTC m=+595.966498737" watchObservedRunningTime="2026-08-04 20:50:48.723233933 +0000 UTC m=+595.966853213"
Aug 04 20:50:48.737299 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.737256    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-vcm4s"]
Aug 04 20:50:48.737510 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.737485    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-vcm4s" podUID="c8c87e14-0506-40ac-818f-ade15a9ac44c" containerName="authorino" containerID="cri-o://c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291" gracePeriod=30
Aug 04 20:50:48.750056 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.750034    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7455c65646-4prrg"]
Aug 04 20:50:48.753660 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.753640    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7455c65646-4prrg"]
Aug 04 20:50:48.978279 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:48.978254    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf"
Aug 04 20:50:49.003015 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.002989    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-vcm4s"
Aug 04 20:50:49.027825 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.027797    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrkdm\" (UniqueName: \"kubernetes.io/projected/f7085976-8975-4356-a3d1-8ca26ae44849-kube-api-access-wrkdm\") pod \"f7085976-8975-4356-a3d1-8ca26ae44849\" (UID: \"f7085976-8975-4356-a3d1-8ca26ae44849\") "
Aug 04 20:50:49.027947 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.027844    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ln646\" (UniqueName: \"kubernetes.io/projected/c8c87e14-0506-40ac-818f-ade15a9ac44c-kube-api-access-ln646\") pod \"c8c87e14-0506-40ac-818f-ade15a9ac44c\" (UID: \"c8c87e14-0506-40ac-818f-ade15a9ac44c\") "
Aug 04 20:50:49.029882 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.029859    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7085976-8975-4356-a3d1-8ca26ae44849-kube-api-access-wrkdm" (OuterVolumeSpecName: "kube-api-access-wrkdm") pod "f7085976-8975-4356-a3d1-8ca26ae44849" (UID: "f7085976-8975-4356-a3d1-8ca26ae44849"). InnerVolumeSpecName "kube-api-access-wrkdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:50:49.029974 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.029897    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8c87e14-0506-40ac-818f-ade15a9ac44c-kube-api-access-ln646" (OuterVolumeSpecName: "kube-api-access-ln646") pod "c8c87e14-0506-40ac-818f-ade15a9ac44c" (UID: "c8c87e14-0506-40ac-818f-ade15a9ac44c"). InnerVolumeSpecName "kube-api-access-ln646". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:50:49.129166 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.129134    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ln646\" (UniqueName: \"kubernetes.io/projected/c8c87e14-0506-40ac-818f-ade15a9ac44c-kube-api-access-ln646\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:50:49.129166 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.129159    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrkdm\" (UniqueName: \"kubernetes.io/projected/f7085976-8975-4356-a3d1-8ca26ae44849-kube-api-access-wrkdm\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:50:49.242782 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.242710    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d034b84-52a9-401f-be06-64fb74d279f8" path="/var/lib/kubelet/pods/9d034b84-52a9-401f-be06-64fb74d279f8/volumes"
Aug 04 20:50:49.699210 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.699172    2563 generic.go:356] "Generic (PLEG): container finished" podID="f7085976-8975-4356-a3d1-8ca26ae44849" containerID="567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd" exitCode=0
Aug 04 20:50:49.699785 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.699231    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf"
Aug 04 20:50:49.699785 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.699265    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf" event={"ID":"f7085976-8975-4356-a3d1-8ca26ae44849","Type":"ContainerDied","Data":"567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd"}
Aug 04 20:50:49.699785 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.699305    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-fxhqf" event={"ID":"f7085976-8975-4356-a3d1-8ca26ae44849","Type":"ContainerDied","Data":"da03fa5f910ad64ac7ea1a1846a5c90cfdecfce541ddafb6c7168bdf5f63ed71"}
Aug 04 20:50:49.699785 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.699350    2563 scope.go:117] "RemoveContainer" containerID="567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd"
Aug 04 20:50:49.700756 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.700653    2563 generic.go:356] "Generic (PLEG): container finished" podID="c8c87e14-0506-40ac-818f-ade15a9ac44c" containerID="c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291" exitCode=0
Aug 04 20:50:49.700756 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.700705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-vcm4s" event={"ID":"c8c87e14-0506-40ac-818f-ade15a9ac44c","Type":"ContainerDied","Data":"c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291"}
Aug 04 20:50:49.700756 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.700748    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-vcm4s" event={"ID":"c8c87e14-0506-40ac-818f-ade15a9ac44c","Type":"ContainerDied","Data":"5cce264340460cc1df03715f90a194948e5a1c7ad715f443ffcd69e806a242dc"}
Aug 04 20:50:49.700914 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.700710    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-vcm4s"
Aug 04 20:50:49.708071 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.708056    2563 scope.go:117] "RemoveContainer" containerID="567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd"
Aug 04 20:50:49.708397 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:50:49.708378    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd\": container with ID starting with 567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd not found: ID does not exist" containerID="567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd"
Aug 04 20:50:49.708510 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.708407    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd"} err="failed to get container status \"567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd\": rpc error: code = NotFound desc = could not find container \"567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd\": container with ID starting with 567fc21c5868e02995b58c70a3dddafbd11ea9b4a4218ae49a91f33fcd0738dd not found: ID does not exist"
Aug 04 20:50:49.708510 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.708436    2563 scope.go:117] "RemoveContainer" containerID="c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291"
Aug 04 20:50:49.716137 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.716120    2563 scope.go:117] "RemoveContainer" containerID="c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291"
Aug 04 20:50:49.716421 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:50:49.716397    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291\": container with ID starting with c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291 not found: ID does not exist" containerID="c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291"
Aug 04 20:50:49.716482 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.716427    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291"} err="failed to get container status \"c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291\": rpc error: code = NotFound desc = could not find container \"c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291\": container with ID starting with c9c0e8b0d196d7b5e4f86e7d7b7019c3eb2e4cad2ae96946fa011977d0c22291 not found: ID does not exist"
Aug 04 20:50:49.722645 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.722624    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxhqf"]
Aug 04 20:50:49.726564 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.726546    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-fxhqf"]
Aug 04 20:50:49.737778 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.737759    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-vcm4s"]
Aug 04 20:50:49.741602 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:49.741584    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-vcm4s"]
Aug 04 20:50:51.243415 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:51.243374    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8c87e14-0506-40ac-818f-ade15a9ac44c" path="/var/lib/kubelet/pods/c8c87e14-0506-40ac-818f-ade15a9ac44c/volumes"
Aug 04 20:50:51.243899 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:51.243818    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7085976-8975-4356-a3d1-8ca26ae44849" path="/var/lib/kubelet/pods/f7085976-8975-4356-a3d1-8ca26ae44849/volumes"
Aug 04 20:50:53.407437 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.407368    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d7f7754c7-7f5jp"]
Aug 04 20:50:53.407754 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.407693    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8c87e14-0506-40ac-818f-ade15a9ac44c" containerName="authorino"
Aug 04 20:50:53.407754 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.407704    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8c87e14-0506-40ac-818f-ade15a9ac44c" containerName="authorino"
Aug 04 20:50:53.407754 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.407721    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7085976-8975-4356-a3d1-8ca26ae44849" containerName="authorino"
Aug 04 20:50:53.407754 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.407726    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7085976-8975-4356-a3d1-8ca26ae44849" containerName="authorino"
Aug 04 20:50:53.407877 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.407777    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8c87e14-0506-40ac-818f-ade15a9ac44c" containerName="authorino"
Aug 04 20:50:53.407877 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.407784    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7085976-8975-4356-a3d1-8ca26ae44849" containerName="authorino"
Aug 04 20:50:53.573749 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.573713    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d7f7754c7-7f5jp"]
Aug 04 20:50:53.573894 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.573824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:50:53.577293 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.577261    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jnwpx\""
Aug 04 20:50:53.577449 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.577303    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 20:50:53.577449 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.577266    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 20:50:53.580085 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.580064    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-8jnvc"]
Aug 04 20:50:53.600481 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.600453    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-8jnvc"]
Aug 04 20:50:53.600619 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.600554    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:50:53.666683 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.666610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj2cn\" (UniqueName: \"kubernetes.io/projected/103ad39c-a4ee-4305-b7d2-20a4d89f193b-kube-api-access-nj2cn\") pod \"maas-controller-5d7f7754c7-7f5jp\" (UID: \"103ad39c-a4ee-4305-b7d2-20a4d89f193b\") " pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:50:53.666683 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.666641    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fhhv\" (UniqueName: \"kubernetes.io/projected/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-kube-api-access-9fhhv\") pod \"maas-controller-7f8b8cf5-8jnvc\" (UID: \"71f0a1c0-7c8e-4b43-af7f-f68be89a9857\") " pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:50:53.666683 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.666684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-webhook-cert\") pod \"maas-controller-7f8b8cf5-8jnvc\" (UID: \"71f0a1c0-7c8e-4b43-af7f-f68be89a9857\") " pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:50:53.666898 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.666758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/103ad39c-a4ee-4305-b7d2-20a4d89f193b-webhook-cert\") pod \"maas-controller-5d7f7754c7-7f5jp\" (UID: \"103ad39c-a4ee-4305-b7d2-20a4d89f193b\") " pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:50:53.767591 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.767562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj2cn\" (UniqueName: \"kubernetes.io/projected/103ad39c-a4ee-4305-b7d2-20a4d89f193b-kube-api-access-nj2cn\") pod \"maas-controller-5d7f7754c7-7f5jp\" (UID: \"103ad39c-a4ee-4305-b7d2-20a4d89f193b\") " pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:50:53.767751 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.767599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fhhv\" (UniqueName: \"kubernetes.io/projected/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-kube-api-access-9fhhv\") pod \"maas-controller-7f8b8cf5-8jnvc\" (UID: \"71f0a1c0-7c8e-4b43-af7f-f68be89a9857\") " pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:50:53.767751 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.767649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-webhook-cert\") pod \"maas-controller-7f8b8cf5-8jnvc\" (UID: \"71f0a1c0-7c8e-4b43-af7f-f68be89a9857\") " pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:50:53.767751 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.767689    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/103ad39c-a4ee-4305-b7d2-20a4d89f193b-webhook-cert\") pod \"maas-controller-5d7f7754c7-7f5jp\" (UID: \"103ad39c-a4ee-4305-b7d2-20a4d89f193b\") " pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:50:53.770015 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.769994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/103ad39c-a4ee-4305-b7d2-20a4d89f193b-webhook-cert\") pod \"maas-controller-5d7f7754c7-7f5jp\" (UID: \"103ad39c-a4ee-4305-b7d2-20a4d89f193b\") " pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:50:53.770181 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.770160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-webhook-cert\") pod \"maas-controller-7f8b8cf5-8jnvc\" (UID: \"71f0a1c0-7c8e-4b43-af7f-f68be89a9857\") " pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:50:53.775904 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.775881    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fhhv\" (UniqueName: \"kubernetes.io/projected/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-kube-api-access-9fhhv\") pod \"maas-controller-7f8b8cf5-8jnvc\" (UID: \"71f0a1c0-7c8e-4b43-af7f-f68be89a9857\") " pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:50:53.776305 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.776284    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj2cn\" (UniqueName: \"kubernetes.io/projected/103ad39c-a4ee-4305-b7d2-20a4d89f193b-kube-api-access-nj2cn\") pod \"maas-controller-5d7f7754c7-7f5jp\" (UID: \"103ad39c-a4ee-4305-b7d2-20a4d89f193b\") " pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:50:53.884030 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.883999    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:50:53.910132 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:53.910107    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:50:54.061523 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:54.061488    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d7f7754c7-7f5jp"]
Aug 04 20:50:54.063249 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:50:54.063213    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103ad39c_a4ee_4305_b7d2_20a4d89f193b.slice/crio-56757d8906866589bc7077fa03dc1ec811a56bff2ffe618b0e65170e24a0a630 WatchSource:0}: Error finding container 56757d8906866589bc7077fa03dc1ec811a56bff2ffe618b0e65170e24a0a630: Status 404 returned error can't find the container with id 56757d8906866589bc7077fa03dc1ec811a56bff2ffe618b0e65170e24a0a630
Aug 04 20:50:54.083080 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:54.083057    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-8jnvc"]
Aug 04 20:50:54.084759 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:50:54.084731    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f0a1c0_7c8e_4b43_af7f_f68be89a9857.slice/crio-bb4c002a7ba9ea9febc53d85a1051efec13e303d27a32a6882c8cad6bebd7eea WatchSource:0}: Error finding container bb4c002a7ba9ea9febc53d85a1051efec13e303d27a32a6882c8cad6bebd7eea: Status 404 returned error can't find the container with id bb4c002a7ba9ea9febc53d85a1051efec13e303d27a32a6882c8cad6bebd7eea
Aug 04 20:50:54.731115 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:54.731074    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc" event={"ID":"71f0a1c0-7c8e-4b43-af7f-f68be89a9857","Type":"ContainerStarted","Data":"bb4c002a7ba9ea9febc53d85a1051efec13e303d27a32a6882c8cad6bebd7eea"}
Aug 04 20:50:54.732421 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:54.732387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp" event={"ID":"103ad39c-a4ee-4305-b7d2-20a4d89f193b","Type":"ContainerStarted","Data":"56757d8906866589bc7077fa03dc1ec811a56bff2ffe618b0e65170e24a0a630"}
Aug 04 20:50:56.746610 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:56.746563    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp" event={"ID":"103ad39c-a4ee-4305-b7d2-20a4d89f193b","Type":"ContainerStarted","Data":"94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28"}
Aug 04 20:50:56.747197 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:56.746670    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:50:56.748601 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:56.748456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc" event={"ID":"71f0a1c0-7c8e-4b43-af7f-f68be89a9857","Type":"ContainerStarted","Data":"2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8"}
Aug 04 20:50:56.748732 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:56.748624    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:50:56.768230 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:56.768176    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp" podStartSLOduration=1.250419932 podStartE2EDuration="3.768160205s" totalImagesPullingTime="2.517740273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:50:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:50:56.766757244 +0000 UTC m=+604.010376550" watchObservedRunningTime="2026-08-04 20:50:56.768160205 +0000 UTC m=+604.011779533"
Aug 04 20:50:56.789911 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:50:56.789851    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc" podStartSLOduration=1.297560393 podStartE2EDuration="3.789831931s" totalImagesPullingTime="2.492271538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:50:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:50:56.787483415 +0000 UTC m=+604.031102695" watchObservedRunningTime="2026-08-04 20:50:56.789831931 +0000 UTC m=+604.033451215"
Aug 04 20:51:07.755608 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:07.755579    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:51:07.756110 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:07.755657    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:51:07.825298 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:07.825266    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d7f7754c7-7f5jp"]
Aug 04 20:51:07.825523 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:07.825499    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp" podUID="103ad39c-a4ee-4305-b7d2-20a4d89f193b" containerName="manager" containerID="cri-o://94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28" gracePeriod=30
Aug 04 20:51:08.065861 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.065832    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:51:08.194846 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.194817    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/103ad39c-a4ee-4305-b7d2-20a4d89f193b-webhook-cert\") pod \"103ad39c-a4ee-4305-b7d2-20a4d89f193b\" (UID: \"103ad39c-a4ee-4305-b7d2-20a4d89f193b\") "
Aug 04 20:51:08.194993 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.194915    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj2cn\" (UniqueName: \"kubernetes.io/projected/103ad39c-a4ee-4305-b7d2-20a4d89f193b-kube-api-access-nj2cn\") pod \"103ad39c-a4ee-4305-b7d2-20a4d89f193b\" (UID: \"103ad39c-a4ee-4305-b7d2-20a4d89f193b\") "
Aug 04 20:51:08.197115 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.197090    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/103ad39c-a4ee-4305-b7d2-20a4d89f193b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "103ad39c-a4ee-4305-b7d2-20a4d89f193b" (UID: "103ad39c-a4ee-4305-b7d2-20a4d89f193b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 20:51:08.197199 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.197108    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/103ad39c-a4ee-4305-b7d2-20a4d89f193b-kube-api-access-nj2cn" (OuterVolumeSpecName: "kube-api-access-nj2cn") pod "103ad39c-a4ee-4305-b7d2-20a4d89f193b" (UID: "103ad39c-a4ee-4305-b7d2-20a4d89f193b"). InnerVolumeSpecName "kube-api-access-nj2cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:51:08.295492 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.295470    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj2cn\" (UniqueName: \"kubernetes.io/projected/103ad39c-a4ee-4305-b7d2-20a4d89f193b-kube-api-access-nj2cn\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:51:08.295592 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.295497    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/103ad39c-a4ee-4305-b7d2-20a4d89f193b-webhook-cert\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:51:08.811993 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.811961    2563 generic.go:356] "Generic (PLEG): container finished" podID="103ad39c-a4ee-4305-b7d2-20a4d89f193b" containerID="94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28" exitCode=0
Aug 04 20:51:08.812483 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.812030    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp"
Aug 04 20:51:08.812483 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.812051    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp" event={"ID":"103ad39c-a4ee-4305-b7d2-20a4d89f193b","Type":"ContainerDied","Data":"94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28"}
Aug 04 20:51:08.812483 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.812103    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d7f7754c7-7f5jp" event={"ID":"103ad39c-a4ee-4305-b7d2-20a4d89f193b","Type":"ContainerDied","Data":"56757d8906866589bc7077fa03dc1ec811a56bff2ffe618b0e65170e24a0a630"}
Aug 04 20:51:08.812483 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.812124    2563 scope.go:117] "RemoveContainer" containerID="94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28"
Aug 04 20:51:08.820927 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.820909    2563 scope.go:117] "RemoveContainer" containerID="94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28"
Aug 04 20:51:08.821174 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:51:08.821159    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28\": container with ID starting with 94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28 not found: ID does not exist" containerID="94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28"
Aug 04 20:51:08.821226 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.821183    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28"} err="failed to get container status \"94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28\": rpc error: code = NotFound desc = could not find container \"94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28\": container with ID starting with 94be594195ec177a590a983ea14e15211c03fe30249bc2afcc0d1ceeb88a2a28 not found: ID does not exist"
Aug 04 20:51:08.835079 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.835060    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d7f7754c7-7f5jp"]
Aug 04 20:51:08.839283 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:08.839265    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5d7f7754c7-7f5jp"]
Aug 04 20:51:09.242775 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:09.242748    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="103ad39c-a4ee-4305-b7d2-20a4d89f193b" path="/var/lib/kubelet/pods/103ad39c-a4ee-4305-b7d2-20a4d89f193b/volumes"
Aug 04 20:51:32.551742 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.551710    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"]
Aug 04 20:51:32.553353 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.552044    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103ad39c-a4ee-4305-b7d2-20a4d89f193b" containerName="manager"
Aug 04 20:51:32.553353 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.552058    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="103ad39c-a4ee-4305-b7d2-20a4d89f193b" containerName="manager"
Aug 04 20:51:32.553353 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.552117    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="103ad39c-a4ee-4305-b7d2-20a4d89f193b" containerName="manager"
Aug 04 20:51:32.554190 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.554175    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:32.557789 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.557770    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q6kj6\""
Aug 04 20:51:32.557898 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.557770    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Aug 04 20:51:32.570525 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.570504    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"]
Aug 04 20:51:32.678587 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.678564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24eb8796-719a-4fbb-aae5-56c09a42bff7-maas-api-tls\") pod \"maas-api-6cf454fbc-wxq2w\" (UID: \"24eb8796-719a-4fbb-aae5-56c09a42bff7\") " pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:32.678732 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.678596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kglqs\" (UniqueName: \"kubernetes.io/projected/24eb8796-719a-4fbb-aae5-56c09a42bff7-kube-api-access-kglqs\") pod \"maas-api-6cf454fbc-wxq2w\" (UID: \"24eb8796-719a-4fbb-aae5-56c09a42bff7\") " pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:32.678732 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.678627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24eb8796-719a-4fbb-aae5-56c09a42bff7-openshift-service-ca\") pod \"maas-api-6cf454fbc-wxq2w\" (UID: \"24eb8796-719a-4fbb-aae5-56c09a42bff7\") " pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:32.779960 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.779930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24eb8796-719a-4fbb-aae5-56c09a42bff7-maas-api-tls\") pod \"maas-api-6cf454fbc-wxq2w\" (UID: \"24eb8796-719a-4fbb-aae5-56c09a42bff7\") " pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:32.780116 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.779979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kglqs\" (UniqueName: \"kubernetes.io/projected/24eb8796-719a-4fbb-aae5-56c09a42bff7-kube-api-access-kglqs\") pod \"maas-api-6cf454fbc-wxq2w\" (UID: \"24eb8796-719a-4fbb-aae5-56c09a42bff7\") " pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:32.780116 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.780007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24eb8796-719a-4fbb-aae5-56c09a42bff7-openshift-service-ca\") pod \"maas-api-6cf454fbc-wxq2w\" (UID: \"24eb8796-719a-4fbb-aae5-56c09a42bff7\") " pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:32.780928 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.780890    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24eb8796-719a-4fbb-aae5-56c09a42bff7-openshift-service-ca\") pod \"maas-api-6cf454fbc-wxq2w\" (UID: \"24eb8796-719a-4fbb-aae5-56c09a42bff7\") " pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:32.782609 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.782589    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24eb8796-719a-4fbb-aae5-56c09a42bff7-maas-api-tls\") pod \"maas-api-6cf454fbc-wxq2w\" (UID: \"24eb8796-719a-4fbb-aae5-56c09a42bff7\") " pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:32.789233 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.789209    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kglqs\" (UniqueName: \"kubernetes.io/projected/24eb8796-719a-4fbb-aae5-56c09a42bff7-kube-api-access-kglqs\") pod \"maas-api-6cf454fbc-wxq2w\" (UID: \"24eb8796-719a-4fbb-aae5-56c09a42bff7\") " pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:32.863632 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:32.863571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:33.005072 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.005042    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"]
Aug 04 20:51:33.006870 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:51:33.006840    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24eb8796_719a_4fbb_aae5_56c09a42bff7.slice/crio-aad3b8684cf039306a7a1a6a8d84167a79c924977055d2245c4bc306f95f73cf WatchSource:0}: Error finding container aad3b8684cf039306a7a1a6a8d84167a79c924977055d2245c4bc306f95f73cf: Status 404 returned error can't find the container with id aad3b8684cf039306a7a1a6a8d84167a79c924977055d2245c4bc306f95f73cf
Aug 04 20:51:33.098843 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.098813    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"]
Aug 04 20:51:33.102021 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.102004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"
Aug 04 20:51:33.104500 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.104477    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Aug 04 20:51:33.104595 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.104578    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-52nh4\""
Aug 04 20:51:33.119727 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.118193    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"]
Aug 04 20:51:33.183964 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.183919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8647a6-432a-4079-830f-ef18a945607b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-822qv\" (UID: \"4a8647a6-432a-4079-830f-ef18a945607b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"
Aug 04 20:51:33.184130 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.183977    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn9hq\" (UniqueName: \"kubernetes.io/projected/4a8647a6-432a-4079-830f-ef18a945607b-kube-api-access-qn9hq\") pod \"payload-pre-processing-687dbb6cf6-822qv\" (UID: \"4a8647a6-432a-4079-830f-ef18a945607b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"
Aug 04 20:51:33.285051 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.285007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn9hq\" (UniqueName: \"kubernetes.io/projected/4a8647a6-432a-4079-830f-ef18a945607b-kube-api-access-qn9hq\") pod \"payload-pre-processing-687dbb6cf6-822qv\" (UID: \"4a8647a6-432a-4079-830f-ef18a945607b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"
Aug 04 20:51:33.285216 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.285102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8647a6-432a-4079-830f-ef18a945607b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-822qv\" (UID: \"4a8647a6-432a-4079-830f-ef18a945607b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"
Aug 04 20:51:33.285773 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.285756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8647a6-432a-4079-830f-ef18a945607b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-822qv\" (UID: \"4a8647a6-432a-4079-830f-ef18a945607b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"
Aug 04 20:51:33.295475 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.295447    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn9hq\" (UniqueName: \"kubernetes.io/projected/4a8647a6-432a-4079-830f-ef18a945607b-kube-api-access-qn9hq\") pod \"payload-pre-processing-687dbb6cf6-822qv\" (UID: \"4a8647a6-432a-4079-830f-ef18a945607b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"
Aug 04 20:51:33.419590 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.419513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"
Aug 04 20:51:33.554902 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.554878    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"]
Aug 04 20:51:33.556654 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:51:33.556628    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8647a6_432a_4079_830f_ef18a945607b.slice/crio-3521bba5b10c8c36a78c54070d39f461a220ea91098d05040922e3f499b46aad WatchSource:0}: Error finding container 3521bba5b10c8c36a78c54070d39f461a220ea91098d05040922e3f499b46aad: Status 404 returned error can't find the container with id 3521bba5b10c8c36a78c54070d39f461a220ea91098d05040922e3f499b46aad
Aug 04 20:51:33.938672 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.938640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv" event={"ID":"4a8647a6-432a-4079-830f-ef18a945607b","Type":"ContainerStarted","Data":"3521bba5b10c8c36a78c54070d39f461a220ea91098d05040922e3f499b46aad"}
Aug 04 20:51:33.939877 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:33.939848    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w" event={"ID":"24eb8796-719a-4fbb-aae5-56c09a42bff7","Type":"ContainerStarted","Data":"aad3b8684cf039306a7a1a6a8d84167a79c924977055d2245c4bc306f95f73cf"}
Aug 04 20:51:34.948271 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:34.948227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w" event={"ID":"24eb8796-719a-4fbb-aae5-56c09a42bff7","Type":"ContainerStarted","Data":"0c1d381a8a6b1defdca73e50981540402d3c80712b79f229c40652172b9c3099"}
Aug 04 20:51:34.948699 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:34.948304    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:34.970435 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:34.970386    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w" podStartSLOduration=1.17869767 podStartE2EDuration="2.970370905s" totalImagesPullingTime="1.791673235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:51:34.968531541 +0000 UTC m=+642.212150827" watchObservedRunningTime="2026-08-04 20:51:34.970370905 +0000 UTC m=+642.213990186"
Aug 04 20:51:35.954677 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:35.954643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv" event={"ID":"4a8647a6-432a-4079-830f-ef18a945607b","Type":"ContainerStarted","Data":"8cb044720159a1b0509a0b85c1aff4ced516b691aa30326a27eb2051372fc74b"}
Aug 04 20:51:35.973456 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:35.973412    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv" podStartSLOduration=1.3194097870000001 podStartE2EDuration="2.97339782s" totalImagesPullingTime="1.653988033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:51:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:51:35.971919703 +0000 UTC m=+643.215538986" watchObservedRunningTime="2026-08-04 20:51:35.97339782 +0000 UTC m=+643.217017124"
Aug 04 20:51:36.959960 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:36.959931    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"
Aug 04 20:51:40.959878 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:40.959842    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6cf454fbc-wxq2w"
Aug 04 20:51:47.966469 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:47.966441    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-822qv"
Aug 04 20:51:55.351115 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.351075    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"]
Aug 04 20:51:55.354689 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.354667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.358234 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.358213    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 04 20:51:55.358374 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.358301    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 04 20:51:55.358441 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.358250    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Aug 04 20:51:55.358517 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.358216    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kftb7\""
Aug 04 20:51:55.369077 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.369024    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"]
Aug 04 20:51:55.464491 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.464457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr4lv\" (UniqueName: \"kubernetes.io/projected/d0587a57-26e0-4696-93af-54049fabc2e3-kube-api-access-cr4lv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.464651 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.464528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.464651 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.464573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0587a57-26e0-4696-93af-54049fabc2e3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.464651 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.464608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.464651 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.464631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.464651 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.464645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.566010 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.565918    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.566010 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.565959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.566010 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.565975    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.566260 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.566010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr4lv\" (UniqueName: \"kubernetes.io/projected/d0587a57-26e0-4696-93af-54049fabc2e3-kube-api-access-cr4lv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.566260 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.566075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.566260 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.566113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0587a57-26e0-4696-93af-54049fabc2e3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.566415 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.566395    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.566471 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.566429    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.566602 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.566582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.568423 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.568400    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0587a57-26e0-4696-93af-54049fabc2e3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.568761 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.568744    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0587a57-26e0-4696-93af-54049fabc2e3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.574870 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.574849    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr4lv\" (UniqueName: \"kubernetes.io/projected/d0587a57-26e0-4696-93af-54049fabc2e3-kube-api-access-cr4lv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8dgj\" (UID: \"d0587a57-26e0-4696-93af-54049fabc2e3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.666333 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.666289    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:51:55.817446 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.817368    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"]
Aug 04 20:51:55.821074 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:51:55.821042    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0587a57_26e0_4696_93af_54049fabc2e3.slice/crio-b51e97ceadab5ac480e384d626759119dff4815b06a3eb1b2222e5b229acbc2a WatchSource:0}: Error finding container b51e97ceadab5ac480e384d626759119dff4815b06a3eb1b2222e5b229acbc2a: Status 404 returned error can't find the container with id b51e97ceadab5ac480e384d626759119dff4815b06a3eb1b2222e5b229acbc2a
Aug 04 20:51:55.822483 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:55.822464    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 20:51:56.060293 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:56.060252    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj" event={"ID":"d0587a57-26e0-4696-93af-54049fabc2e3","Type":"ContainerStarted","Data":"b51e97ceadab5ac480e384d626759119dff4815b06a3eb1b2222e5b229acbc2a"}
Aug 04 20:51:57.066498 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:57.066455    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj" event={"ID":"d0587a57-26e0-4696-93af-54049fabc2e3","Type":"ContainerStarted","Data":"6e0a082cbdf33a234089eca354c7c53c9bf349c8f26e3ce363bb5fbc343f30d8"}
Aug 04 20:51:57.085488 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:51:57.085444    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj" podStartSLOduration=1.067357643 podStartE2EDuration="2.085429334s" totalImagesPullingTime="1.018071691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:51:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:51:57.084660937 +0000 UTC m=+664.328280245" watchObservedRunningTime="2026-08-04 20:51:57.085429334 +0000 UTC m=+664.329048615"
Aug 04 20:52:05.667356 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:05.667299    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:52:05.667819 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:05.667404    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:52:05.679744 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:05.679719    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:52:06.124031 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:06.124006    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8dgj"
Aug 04 20:52:23.758666 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.758629    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"]
Aug 04 20:52:23.762040 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.762015    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.764609 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.764589    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Aug 04 20:52:23.772666 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.772643    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"]
Aug 04 20:52:23.896196 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.896160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.896196 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.896195    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42e1d088-a11f-40ba-a522-04437b8a7f34-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.896462 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.896232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lcr8\" (UniqueName: \"kubernetes.io/projected/42e1d088-a11f-40ba-a522-04437b8a7f34-kube-api-access-2lcr8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.896462 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.896283    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.896462 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.896300    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.896462 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.896342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.997502 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.997464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.997660 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.997507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42e1d088-a11f-40ba-a522-04437b8a7f34-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.997660 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.997559    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lcr8\" (UniqueName: \"kubernetes.io/projected/42e1d088-a11f-40ba-a522-04437b8a7f34-kube-api-access-2lcr8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.997660 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.997593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.997660 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.997616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.997660 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.997643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.998066 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.997918    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.998066 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.997963    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:23.998066 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:23.998005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:24.000227 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:24.000194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42e1d088-a11f-40ba-a522-04437b8a7f34-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:24.000649 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:24.000630    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42e1d088-a11f-40ba-a522-04437b8a7f34-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:24.006428 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:24.006403    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lcr8\" (UniqueName: \"kubernetes.io/projected/42e1d088-a11f-40ba-a522-04437b8a7f34-kube-api-access-2lcr8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6pvp9\" (UID: \"42e1d088-a11f-40ba-a522-04437b8a7f34\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:24.073279 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:24.073223    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:24.208001 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:24.207979    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"]
Aug 04 20:52:24.209711 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:52:24.209686    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e1d088_a11f_40ba_a522_04437b8a7f34.slice/crio-e6568a246090b51cd0b80b6e34adc2091a59bc61e1c0301861aae50fb62b44a2 WatchSource:0}: Error finding container e6568a246090b51cd0b80b6e34adc2091a59bc61e1c0301861aae50fb62b44a2: Status 404 returned error can't find the container with id e6568a246090b51cd0b80b6e34adc2091a59bc61e1c0301861aae50fb62b44a2
Aug 04 20:52:25.212220 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:25.212189    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9" event={"ID":"42e1d088-a11f-40ba-a522-04437b8a7f34","Type":"ContainerStarted","Data":"46cb5b880e5b8d90848ec176958a1881a9e5ba6ea9d8539c81ceda1fc4636ef7"}
Aug 04 20:52:25.212604 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:25.212235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9" event={"ID":"42e1d088-a11f-40ba-a522-04437b8a7f34","Type":"ContainerStarted","Data":"e6568a246090b51cd0b80b6e34adc2091a59bc61e1c0301861aae50fb62b44a2"}
Aug 04 20:52:25.233227 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:25.233178    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9" podStartSLOduration=2.058585342 podStartE2EDuration="2.233164061s" totalImagesPullingTime="174.578719ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:52:25.230861123 +0000 UTC m=+692.474480404" watchObservedRunningTime="2026-08-04 20:52:25.233164061 +0000 UTC m=+692.476783342"
Aug 04 20:52:30.656566 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.656535    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"]
Aug 04 20:52:30.660699 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.660680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.663201 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.663181    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Aug 04 20:52:30.671108 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.671089    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"]
Aug 04 20:52:30.756116 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.756087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh8zf\" (UniqueName: \"kubernetes.io/projected/4829dc1f-2116-4de9-9472-be6f00f7ec5b-kube-api-access-mh8zf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.756116 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.756117    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.756355 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.756134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.756355 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.756161    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4829dc1f-2116-4de9-9472-be6f00f7ec5b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.756355 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.756254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.756355 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.756306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.857522 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.857491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh8zf\" (UniqueName: \"kubernetes.io/projected/4829dc1f-2116-4de9-9472-be6f00f7ec5b-kube-api-access-mh8zf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.857522 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.857522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.857729 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.857538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.857729 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.857562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4829dc1f-2116-4de9-9472-be6f00f7ec5b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.857729 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.857589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.857729 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.857622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.857971 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.857947    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.858032 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.857992    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.858032 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.858017    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.859958 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.859935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4829dc1f-2116-4de9-9472-be6f00f7ec5b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.860091 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.860075    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4829dc1f-2116-4de9-9472-be6f00f7ec5b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.867429 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.867397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh8zf\" (UniqueName: \"kubernetes.io/projected/4829dc1f-2116-4de9-9472-be6f00f7ec5b-kube-api-access-mh8zf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k74rf\" (UID: \"4829dc1f-2116-4de9-9472-be6f00f7ec5b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:30.970366 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:30.970291    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:31.112944 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:31.112921    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"]
Aug 04 20:52:31.114825 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:52:31.114767    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4829dc1f_2116_4de9_9472_be6f00f7ec5b.slice/crio-980f5ad5bf5bae63d590da0a051f233ed4d2683813fd5d4d844898922fc74fec WatchSource:0}: Error finding container 980f5ad5bf5bae63d590da0a051f233ed4d2683813fd5d4d844898922fc74fec: Status 404 returned error can't find the container with id 980f5ad5bf5bae63d590da0a051f233ed4d2683813fd5d4d844898922fc74fec
Aug 04 20:52:31.246097 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:31.246017    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf" event={"ID":"4829dc1f-2116-4de9-9472-be6f00f7ec5b","Type":"ContainerStarted","Data":"980f5ad5bf5bae63d590da0a051f233ed4d2683813fd5d4d844898922fc74fec"}
Aug 04 20:52:32.252983 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:32.252953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf" event={"ID":"4829dc1f-2116-4de9-9472-be6f00f7ec5b","Type":"ContainerStarted","Data":"c82496f032982b8e81eb1fab714442d75d6906e2e319234a402bed8c1676ad6f"}
Aug 04 20:52:32.271850 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:32.271808    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf" podStartSLOduration=2.066806259 podStartE2EDuration="2.271794046s" totalImagesPullingTime="204.987787ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:52:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:52:32.270354143 +0000 UTC m=+699.513973427" watchObservedRunningTime="2026-08-04 20:52:32.271794046 +0000 UTC m=+699.515413326"
Aug 04 20:52:34.074057 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:34.074023    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:34.074514 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:34.074352    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:34.087447 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:34.087422    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:34.275520 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:34.275494    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6pvp9"
Aug 04 20:52:40.970873 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:40.970837    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:40.971440 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:40.970893    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:40.983063 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:40.983042    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:52:41.316691 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:52:41.316609    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k74rf"
Aug 04 20:53:18.529743 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:18.529661    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c5c755469-flqx5"]
Aug 04 20:53:18.533238 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:18.533218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c5c755469-flqx5"
Aug 04 20:53:18.541260 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:18.541236    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c5c755469-flqx5"]
Aug 04 20:53:18.627064 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:18.627021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a25d6c26-82d0-4529-a211-97fc6ba88095-tls-cert\") pod \"authorino-7c5c755469-flqx5\" (UID: \"a25d6c26-82d0-4529-a211-97fc6ba88095\") " pod="kuadrant-system/authorino-7c5c755469-flqx5"
Aug 04 20:53:18.627234 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:18.627073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqw8w\" (UniqueName: \"kubernetes.io/projected/a25d6c26-82d0-4529-a211-97fc6ba88095-kube-api-access-vqw8w\") pod \"authorino-7c5c755469-flqx5\" (UID: \"a25d6c26-82d0-4529-a211-97fc6ba88095\") " pod="kuadrant-system/authorino-7c5c755469-flqx5"
Aug 04 20:53:18.727517 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:18.727487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a25d6c26-82d0-4529-a211-97fc6ba88095-tls-cert\") pod \"authorino-7c5c755469-flqx5\" (UID: \"a25d6c26-82d0-4529-a211-97fc6ba88095\") " pod="kuadrant-system/authorino-7c5c755469-flqx5"
Aug 04 20:53:18.727657 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:18.727523    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqw8w\" (UniqueName: \"kubernetes.io/projected/a25d6c26-82d0-4529-a211-97fc6ba88095-kube-api-access-vqw8w\") pod \"authorino-7c5c755469-flqx5\" (UID: \"a25d6c26-82d0-4529-a211-97fc6ba88095\") " pod="kuadrant-system/authorino-7c5c755469-flqx5"
Aug 04 20:53:18.729952 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:18.729931    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a25d6c26-82d0-4529-a211-97fc6ba88095-tls-cert\") pod \"authorino-7c5c755469-flqx5\" (UID: \"a25d6c26-82d0-4529-a211-97fc6ba88095\") " pod="kuadrant-system/authorino-7c5c755469-flqx5"
Aug 04 20:53:18.736598 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:18.736578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqw8w\" (UniqueName: \"kubernetes.io/projected/a25d6c26-82d0-4529-a211-97fc6ba88095-kube-api-access-vqw8w\") pod \"authorino-7c5c755469-flqx5\" (UID: \"a25d6c26-82d0-4529-a211-97fc6ba88095\") " pod="kuadrant-system/authorino-7c5c755469-flqx5"
Aug 04 20:53:18.844247 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:18.844172    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c5c755469-flqx5"
Aug 04 20:53:18.983984 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:18.983948    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c5c755469-flqx5"]
Aug 04 20:53:18.988211 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:53:18.988186    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda25d6c26_82d0_4529_a211_97fc6ba88095.slice/crio-76b84680a14619fcb4aeb3b318c6333263b55c243c26f3932776e49cc4037ab0 WatchSource:0}: Error finding container 76b84680a14619fcb4aeb3b318c6333263b55c243c26f3932776e49cc4037ab0: Status 404 returned error can't find the container with id 76b84680a14619fcb4aeb3b318c6333263b55c243c26f3932776e49cc4037ab0
Aug 04 20:53:19.510599 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:19.510565    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c5c755469-flqx5" event={"ID":"a25d6c26-82d0-4529-a211-97fc6ba88095","Type":"ContainerStarted","Data":"76b84680a14619fcb4aeb3b318c6333263b55c243c26f3932776e49cc4037ab0"}
Aug 04 20:53:20.519503 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:20.519464    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c5c755469-flqx5" event={"ID":"a25d6c26-82d0-4529-a211-97fc6ba88095","Type":"ContainerStarted","Data":"64b576c3ced1bfddd0692ea4b9c1c3503b88a5e332f88b24879a63f1db9db449"}
Aug 04 20:53:20.539079 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:20.539024    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c5c755469-flqx5" podStartSLOduration=1.844255082 podStartE2EDuration="2.539006362s" totalImagesPullingTime="694.75128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:53:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:53:20.536421773 +0000 UTC m=+747.780041047" watchObservedRunningTime="2026-08-04 20:53:20.539006362 +0000 UTC m=+747.782625649"
Aug 04 20:53:20.576389 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:20.576355    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79d565c59b-24pvp"]
Aug 04 20:53:20.579334 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:20.577209    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-79d565c59b-24pvp" podUID="397210c0-6737-4bf3-84c7-ebfa58c5d355" containerName="authorino" containerID="cri-o://75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3" gracePeriod=30
Aug 04 20:53:20.850129 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:20.850094    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79d565c59b-24pvp"
Aug 04 20:53:20.949773 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:20.949744    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/397210c0-6737-4bf3-84c7-ebfa58c5d355-tls-cert\") pod \"397210c0-6737-4bf3-84c7-ebfa58c5d355\" (UID: \"397210c0-6737-4bf3-84c7-ebfa58c5d355\") "
Aug 04 20:53:20.949974 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:20.949843    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7vp9\" (UniqueName: \"kubernetes.io/projected/397210c0-6737-4bf3-84c7-ebfa58c5d355-kube-api-access-p7vp9\") pod \"397210c0-6737-4bf3-84c7-ebfa58c5d355\" (UID: \"397210c0-6737-4bf3-84c7-ebfa58c5d355\") "
Aug 04 20:53:20.952301 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:20.952272    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/397210c0-6737-4bf3-84c7-ebfa58c5d355-kube-api-access-p7vp9" (OuterVolumeSpecName: "kube-api-access-p7vp9") pod "397210c0-6737-4bf3-84c7-ebfa58c5d355" (UID: "397210c0-6737-4bf3-84c7-ebfa58c5d355"). InnerVolumeSpecName "kube-api-access-p7vp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:53:20.960868 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:20.960770    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/397210c0-6737-4bf3-84c7-ebfa58c5d355-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "397210c0-6737-4bf3-84c7-ebfa58c5d355" (UID: "397210c0-6737-4bf3-84c7-ebfa58c5d355"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 20:53:21.050540 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:21.050511    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7vp9\" (UniqueName: \"kubernetes.io/projected/397210c0-6737-4bf3-84c7-ebfa58c5d355-kube-api-access-p7vp9\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:53:21.050540 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:21.050538    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/397210c0-6737-4bf3-84c7-ebfa58c5d355-tls-cert\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:53:21.527920 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:21.527833    2563 generic.go:356] "Generic (PLEG): container finished" podID="397210c0-6737-4bf3-84c7-ebfa58c5d355" containerID="75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3" exitCode=0
Aug 04 20:53:21.527920 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:21.527880    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79d565c59b-24pvp"
Aug 04 20:53:21.528466 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:21.527935    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79d565c59b-24pvp" event={"ID":"397210c0-6737-4bf3-84c7-ebfa58c5d355","Type":"ContainerDied","Data":"75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3"}
Aug 04 20:53:21.528466 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:21.527980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79d565c59b-24pvp" event={"ID":"397210c0-6737-4bf3-84c7-ebfa58c5d355","Type":"ContainerDied","Data":"eca987c52e85412e36d5f00d50b79bac1b81a671f7b85065f13c17b25c747bc9"}
Aug 04 20:53:21.528466 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:21.528004    2563 scope.go:117] "RemoveContainer" containerID="75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3"
Aug 04 20:53:21.536829 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:21.536814    2563 scope.go:117] "RemoveContainer" containerID="75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3"
Aug 04 20:53:21.537068 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:53:21.537052    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3\": container with ID starting with 75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3 not found: ID does not exist" containerID="75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3"
Aug 04 20:53:21.537137 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:21.537081    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3"} err="failed to get container status \"75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3\": rpc error: code = NotFound desc = could not find container \"75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3\": container with ID starting with 75a0bd91f765d0e6335c01b3296751e2cf026a280b305b4fbcd7869dee4961f3 not found: ID does not exist"
Aug 04 20:53:21.550791 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:21.550768    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79d565c59b-24pvp"]
Aug 04 20:53:21.554928 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:21.554897    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79d565c59b-24pvp"]
Aug 04 20:53:23.243140 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:53:23.243108    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="397210c0-6737-4bf3-84c7-ebfa58c5d355" path="/var/lib/kubelet/pods/397210c0-6737-4bf3-84c7-ebfa58c5d355/volumes"
Aug 04 20:54:09.179028 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.178994    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-8jnvc"]
Aug 04 20:54:09.179540 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.179269    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc" podUID="71f0a1c0-7c8e-4b43-af7f-f68be89a9857" containerName="manager" containerID="cri-o://2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8" gracePeriod=30
Aug 04 20:54:09.434372 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.434302    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:54:09.576209 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.576175    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-webhook-cert\") pod \"71f0a1c0-7c8e-4b43-af7f-f68be89a9857\" (UID: \"71f0a1c0-7c8e-4b43-af7f-f68be89a9857\") "
Aug 04 20:54:09.576414 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.576219    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fhhv\" (UniqueName: \"kubernetes.io/projected/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-kube-api-access-9fhhv\") pod \"71f0a1c0-7c8e-4b43-af7f-f68be89a9857\" (UID: \"71f0a1c0-7c8e-4b43-af7f-f68be89a9857\") "
Aug 04 20:54:09.578281 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.578249    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "71f0a1c0-7c8e-4b43-af7f-f68be89a9857" (UID: "71f0a1c0-7c8e-4b43-af7f-f68be89a9857"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 20:54:09.578281 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.578272    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-kube-api-access-9fhhv" (OuterVolumeSpecName: "kube-api-access-9fhhv") pod "71f0a1c0-7c8e-4b43-af7f-f68be89a9857" (UID: "71f0a1c0-7c8e-4b43-af7f-f68be89a9857"). InnerVolumeSpecName "kube-api-access-9fhhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:54:09.677454 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.677431    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-webhook-cert\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:54:09.677454 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.677451    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fhhv\" (UniqueName: \"kubernetes.io/projected/71f0a1c0-7c8e-4b43-af7f-f68be89a9857-kube-api-access-9fhhv\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:54:09.791562 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.791501    2563 generic.go:356] "Generic (PLEG): container finished" podID="71f0a1c0-7c8e-4b43-af7f-f68be89a9857" containerID="2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8" exitCode=0
Aug 04 20:54:09.791562 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.791555    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc"
Aug 04 20:54:09.791702 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.791584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc" event={"ID":"71f0a1c0-7c8e-4b43-af7f-f68be89a9857","Type":"ContainerDied","Data":"2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8"}
Aug 04 20:54:09.791702 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.791637    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8b8cf5-8jnvc" event={"ID":"71f0a1c0-7c8e-4b43-af7f-f68be89a9857","Type":"ContainerDied","Data":"bb4c002a7ba9ea9febc53d85a1051efec13e303d27a32a6882c8cad6bebd7eea"}
Aug 04 20:54:09.791702 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.791659    2563 scope.go:117] "RemoveContainer" containerID="2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8"
Aug 04 20:54:09.800504 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.800487    2563 scope.go:117] "RemoveContainer" containerID="2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8"
Aug 04 20:54:09.800866 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:54:09.800771    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8\": container with ID starting with 2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8 not found: ID does not exist" containerID="2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8"
Aug 04 20:54:09.800866 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.800796    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8"} err="failed to get container status \"2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8\": rpc error: code = NotFound desc = could not find container \"2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8\": container with ID starting with 2555225adf224985e4db70621461332d5f520d280e809e52959dc1b15cede2a8 not found: ID does not exist"
Aug 04 20:54:09.815205 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.815187    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-8jnvc"]
Aug 04 20:54:09.818908 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:09.818882    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-8jnvc"]
Aug 04 20:54:11.243358 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:54:11.243308    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71f0a1c0-7c8e-4b43-af7f-f68be89a9857" path="/var/lib/kubelet/pods/71f0a1c0-7c8e-4b43-af7f-f68be89a9857/volumes"
Aug 04 20:55:10.607185 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.607156    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-hvzbx"]
Aug 04 20:55:10.607683 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.607513    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="397210c0-6737-4bf3-84c7-ebfa58c5d355" containerName="authorino"
Aug 04 20:55:10.607683 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.607524    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="397210c0-6737-4bf3-84c7-ebfa58c5d355" containerName="authorino"
Aug 04 20:55:10.607683 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.607548    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71f0a1c0-7c8e-4b43-af7f-f68be89a9857" containerName="manager"
Aug 04 20:55:10.607683 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.607553    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71f0a1c0-7c8e-4b43-af7f-f68be89a9857" containerName="manager"
Aug 04 20:55:10.607683 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.607616    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="71f0a1c0-7c8e-4b43-af7f-f68be89a9857" containerName="manager"
Aug 04 20:55:10.607683 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.607628    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="397210c0-6737-4bf3-84c7-ebfa58c5d355" containerName="authorino"
Aug 04 20:55:10.611768 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.611751    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:10.615111 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.615089    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 20:55:10.615245 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.615116    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jnwpx\""
Aug 04 20:55:10.615245 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.615117    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 20:55:10.619159 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.618844    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-hvzbx"]
Aug 04 20:55:10.630215 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.630182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84l4s\" (UniqueName: \"kubernetes.io/projected/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-kube-api-access-84l4s\") pod \"maas-controller-7f8b8cf5-hvzbx\" (UID: \"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568\") " pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:10.630333 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.630228    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-webhook-cert\") pod \"maas-controller-7f8b8cf5-hvzbx\" (UID: \"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568\") " pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:10.730820 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.730794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84l4s\" (UniqueName: \"kubernetes.io/projected/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-kube-api-access-84l4s\") pod \"maas-controller-7f8b8cf5-hvzbx\" (UID: \"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568\") " pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:10.730820 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.730829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-webhook-cert\") pod \"maas-controller-7f8b8cf5-hvzbx\" (UID: \"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568\") " pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:10.733239 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.733219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-webhook-cert\") pod \"maas-controller-7f8b8cf5-hvzbx\" (UID: \"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568\") " pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:10.738679 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.738652    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84l4s\" (UniqueName: \"kubernetes.io/projected/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-kube-api-access-84l4s\") pod \"maas-controller-7f8b8cf5-hvzbx\" (UID: \"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568\") " pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:10.923030 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:10.923010    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:11.258508 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:11.258482    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-hvzbx"]
Aug 04 20:55:11.259948 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:55:11.259919    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2fd465_ea55_4b66_a9c6_7f00bf2d8568.slice/crio-9417674fa0a90ffa105f379ad3e03a6aaa263ec3c6815703ba37a9396cf01e18 WatchSource:0}: Error finding container 9417674fa0a90ffa105f379ad3e03a6aaa263ec3c6815703ba37a9396cf01e18: Status 404 returned error can't find the container with id 9417674fa0a90ffa105f379ad3e03a6aaa263ec3c6815703ba37a9396cf01e18
Aug 04 20:55:12.117010 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:12.116929    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx" event={"ID":"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568","Type":"ContainerStarted","Data":"009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6"}
Aug 04 20:55:12.117010 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:12.116984    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:12.117010 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:12.117004    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx" event={"ID":"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568","Type":"ContainerStarted","Data":"9417674fa0a90ffa105f379ad3e03a6aaa263ec3c6815703ba37a9396cf01e18"}
Aug 04 20:55:12.135585 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:12.135541    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx" podStartSLOduration=1.723895872 podStartE2EDuration="2.135528444s" totalImagesPullingTime="411.632572ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:55:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:55:12.133342657 +0000 UTC m=+859.376961937" watchObservedRunningTime="2026-08-04 20:55:12.135528444 +0000 UTC m=+859.379147726"
Aug 04 20:55:23.124593 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:23.124563    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:27.757794 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:27.757764    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-hvzbx"]
Aug 04 20:55:27.758342 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:27.758045    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx" podUID="3e2fd465-ea55-4b66-a9c6-7f00bf2d8568" containerName="manager" containerID="cri-o://009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6" gracePeriod=30
Aug 04 20:55:28.002464 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.002443    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:28.065691 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.065609    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84l4s\" (UniqueName: \"kubernetes.io/projected/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-kube-api-access-84l4s\") pod \"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568\" (UID: \"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568\") "
Aug 04 20:55:28.065691 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.065678    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-webhook-cert\") pod \"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568\" (UID: \"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568\") "
Aug 04 20:55:28.067771 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.067746    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-kube-api-access-84l4s" (OuterVolumeSpecName: "kube-api-access-84l4s") pod "3e2fd465-ea55-4b66-a9c6-7f00bf2d8568" (UID: "3e2fd465-ea55-4b66-a9c6-7f00bf2d8568"). InnerVolumeSpecName "kube-api-access-84l4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 20:55:28.067871 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.067773    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3e2fd465-ea55-4b66-a9c6-7f00bf2d8568" (UID: "3e2fd465-ea55-4b66-a9c6-7f00bf2d8568"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 20:55:28.166324 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.166295    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84l4s\" (UniqueName: \"kubernetes.io/projected/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-kube-api-access-84l4s\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:55:28.166422 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.166335    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568-webhook-cert\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 20:55:28.201925 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.201897    2563 generic.go:356] "Generic (PLEG): container finished" podID="3e2fd465-ea55-4b66-a9c6-7f00bf2d8568" containerID="009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6" exitCode=0
Aug 04 20:55:28.202005 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.201954    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx"
Aug 04 20:55:28.202005 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.201978    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx" event={"ID":"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568","Type":"ContainerDied","Data":"009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6"}
Aug 04 20:55:28.202091 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.202034    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8b8cf5-hvzbx" event={"ID":"3e2fd465-ea55-4b66-a9c6-7f00bf2d8568","Type":"ContainerDied","Data":"9417674fa0a90ffa105f379ad3e03a6aaa263ec3c6815703ba37a9396cf01e18"}
Aug 04 20:55:28.202091 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.202058    2563 scope.go:117] "RemoveContainer" containerID="009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6"
Aug 04 20:55:28.211159 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.211141    2563 scope.go:117] "RemoveContainer" containerID="009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6"
Aug 04 20:55:28.211444 ip-10-0-129-249 kubenswrapper[2563]: E0804 20:55:28.211426    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6\": container with ID starting with 009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6 not found: ID does not exist" containerID="009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6"
Aug 04 20:55:28.211514 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.211455    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6"} err="failed to get container status \"009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6\": rpc error: code = NotFound desc = could not find container \"009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6\": container with ID starting with 009172ab826cc1c874639f68788fc884ff08f2e24d5ca0f9a3e945052a7f4ec6 not found: ID does not exist"
Aug 04 20:55:28.222743 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.222725    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-hvzbx"]
Aug 04 20:55:28.226535 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:28.226515    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-hvzbx"]
Aug 04 20:55:29.005562 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.005529    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-fzrmj"]
Aug 04 20:55:29.005963 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.005898    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e2fd465-ea55-4b66-a9c6-7f00bf2d8568" containerName="manager"
Aug 04 20:55:29.005963 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.005910    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e2fd465-ea55-4b66-a9c6-7f00bf2d8568" containerName="manager"
Aug 04 20:55:29.006038 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.005989    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e2fd465-ea55-4b66-a9c6-7f00bf2d8568" containerName="manager"
Aug 04 20:55:29.010170 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.010148    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8b8cf5-fzrmj"
Aug 04 20:55:29.013731 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.013715    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 20:55:29.013731 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.013719    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jnwpx\""
Aug 04 20:55:29.013731 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.013723    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 20:55:29.019551 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.019522    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-fzrmj"]
Aug 04 20:55:29.072361 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.072310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x29bv\" (UniqueName: \"kubernetes.io/projected/629e6a83-3e5e-4e36-894a-b2b72bb04ccc-kube-api-access-x29bv\") pod \"maas-controller-7f8b8cf5-fzrmj\" (UID: \"629e6a83-3e5e-4e36-894a-b2b72bb04ccc\") " pod="opendatahub/maas-controller-7f8b8cf5-fzrmj"
Aug 04 20:55:29.072473 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.072417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/629e6a83-3e5e-4e36-894a-b2b72bb04ccc-webhook-cert\") pod \"maas-controller-7f8b8cf5-fzrmj\" (UID: \"629e6a83-3e5e-4e36-894a-b2b72bb04ccc\") " pod="opendatahub/maas-controller-7f8b8cf5-fzrmj"
Aug 04 20:55:29.173440 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.173416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x29bv\" (UniqueName: \"kubernetes.io/projected/629e6a83-3e5e-4e36-894a-b2b72bb04ccc-kube-api-access-x29bv\") pod \"maas-controller-7f8b8cf5-fzrmj\" (UID: \"629e6a83-3e5e-4e36-894a-b2b72bb04ccc\") " pod="opendatahub/maas-controller-7f8b8cf5-fzrmj"
Aug 04 20:55:29.173551 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.173453    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/629e6a83-3e5e-4e36-894a-b2b72bb04ccc-webhook-cert\") pod \"maas-controller-7f8b8cf5-fzrmj\" (UID: \"629e6a83-3e5e-4e36-894a-b2b72bb04ccc\") " pod="opendatahub/maas-controller-7f8b8cf5-fzrmj"
Aug 04 20:55:29.175828 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.175810    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/629e6a83-3e5e-4e36-894a-b2b72bb04ccc-webhook-cert\") pod \"maas-controller-7f8b8cf5-fzrmj\" (UID: \"629e6a83-3e5e-4e36-894a-b2b72bb04ccc\") " pod="opendatahub/maas-controller-7f8b8cf5-fzrmj"
Aug 04 20:55:29.182353 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.182306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x29bv\" (UniqueName: \"kubernetes.io/projected/629e6a83-3e5e-4e36-894a-b2b72bb04ccc-kube-api-access-x29bv\") pod \"maas-controller-7f8b8cf5-fzrmj\" (UID: \"629e6a83-3e5e-4e36-894a-b2b72bb04ccc\") " pod="opendatahub/maas-controller-7f8b8cf5-fzrmj"
Aug 04 20:55:29.243235 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.243214    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e2fd465-ea55-4b66-a9c6-7f00bf2d8568" path="/var/lib/kubelet/pods/3e2fd465-ea55-4b66-a9c6-7f00bf2d8568/volumes"
Aug 04 20:55:29.322658 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.322611    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8b8cf5-fzrmj"
Aug 04 20:55:29.458038 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:29.458016    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f8b8cf5-fzrmj"]
Aug 04 20:55:29.458953 ip-10-0-129-249 kubenswrapper[2563]: W0804 20:55:29.458918    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629e6a83_3e5e_4e36_894a_b2b72bb04ccc.slice/crio-fbc34a10637c0a9a0a6975ff66414679ffb2976db3a5edd01199137a12f946a4 WatchSource:0}: Error finding container fbc34a10637c0a9a0a6975ff66414679ffb2976db3a5edd01199137a12f946a4: Status 404 returned error can't find the container with id fbc34a10637c0a9a0a6975ff66414679ffb2976db3a5edd01199137a12f946a4
Aug 04 20:55:30.215402 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:30.215303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8b8cf5-fzrmj" event={"ID":"629e6a83-3e5e-4e36-894a-b2b72bb04ccc","Type":"ContainerStarted","Data":"40f884b13f2312fa86a7f81ef92ea5f224c5488800dc02979e0efd65a48fbd52"}
Aug 04 20:55:30.215402 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:30.215370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8b8cf5-fzrmj" event={"ID":"629e6a83-3e5e-4e36-894a-b2b72bb04ccc","Type":"ContainerStarted","Data":"fbc34a10637c0a9a0a6975ff66414679ffb2976db3a5edd01199137a12f946a4"}
Aug 04 20:55:30.215793 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:30.215460    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f8b8cf5-fzrmj"
Aug 04 20:55:30.233207 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:30.233163    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f8b8cf5-fzrmj" podStartSLOduration=1.749093979 podStartE2EDuration="2.23314856s" totalImagesPullingTime="484.054581ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 20:55:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 20:55:30.232365714 +0000 UTC m=+877.475984996" watchObservedRunningTime="2026-08-04 20:55:30.23314856 +0000 UTC m=+877.476767841"
Aug 04 20:55:41.222889 ip-10-0-129-249 kubenswrapper[2563]: I0804 20:55:41.222858    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f8b8cf5-fzrmj"
Aug 04 21:01:56.574950 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:56.574860    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"]
Aug 04 21:01:56.575453 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:56.575114    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx" podUID="d8ad76b9-d8e9-4fb3-835d-8554e7325ea0" containerName="manager" containerID="cri-o://99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353" gracePeriod=10
Aug 04 21:01:56.827591 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:56.827532    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 21:01:56.861431 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:56.861404    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-extensions-socket-volume\") pod \"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0\" (UID: \"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0\") "
Aug 04 21:01:56.861589 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:56.861446    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh5jw\" (UniqueName: \"kubernetes.io/projected/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-kube-api-access-dh5jw\") pod \"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0\" (UID: \"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0\") "
Aug 04 21:01:56.861882 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:56.861844    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d8ad76b9-d8e9-4fb3-835d-8554e7325ea0" (UID: "d8ad76b9-d8e9-4fb3-835d-8554e7325ea0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 21:01:56.863676 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:56.863647    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-kube-api-access-dh5jw" (OuterVolumeSpecName: "kube-api-access-dh5jw") pod "d8ad76b9-d8e9-4fb3-835d-8554e7325ea0" (UID: "d8ad76b9-d8e9-4fb3-835d-8554e7325ea0"). InnerVolumeSpecName "kube-api-access-dh5jw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:01:56.962702 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:56.962667    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-extensions-socket-volume\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:01:56.962702 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:56.962697    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh5jw\" (UniqueName: \"kubernetes.io/projected/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0-kube-api-access-dh5jw\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:01:57.281691 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:57.281651    2563 generic.go:356] "Generic (PLEG): container finished" podID="d8ad76b9-d8e9-4fb3-835d-8554e7325ea0" containerID="99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353" exitCode=0
Aug 04 21:01:57.281858 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:57.281696    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"
Aug 04 21:01:57.281858 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:57.281749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx" event={"ID":"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0","Type":"ContainerDied","Data":"99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353"}
Aug 04 21:01:57.281858 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:57.281800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx" event={"ID":"d8ad76b9-d8e9-4fb3-835d-8554e7325ea0","Type":"ContainerDied","Data":"26a8555ab65e75c3e02964f072ad170770de6c32a182ea181508056547dd7a43"}
Aug 04 21:01:57.281858 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:57.281824    2563 scope.go:117] "RemoveContainer" containerID="99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353"
Aug 04 21:01:57.292396 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:57.292376    2563 scope.go:117] "RemoveContainer" containerID="99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353"
Aug 04 21:01:57.292683 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:01:57.292664    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353\": container with ID starting with 99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353 not found: ID does not exist" containerID="99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353"
Aug 04 21:01:57.292727 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:57.292691    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353"} err="failed to get container status \"99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353\": rpc error: code = NotFound desc = could not find container \"99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353\": container with ID starting with 99021358366c7634c07a3a26fd681e719b95af38e90fb7eb8e64bbc3aa6f7353 not found: ID does not exist"
Aug 04 21:01:57.301388 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:57.301368    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"]
Aug 04 21:01:57.306453 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:57.306434    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-cmsrx"]
Aug 04 21:01:59.243141 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:01:59.243110    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8ad76b9-d8e9-4fb3-835d-8554e7325ea0" path="/var/lib/kubelet/pods/d8ad76b9-d8e9-4fb3-835d-8554e7325ea0/volumes"
Aug 04 21:02:57.289504 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.289469    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"]
Aug 04 21:02:57.291816 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.289870    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8ad76b9-d8e9-4fb3-835d-8554e7325ea0" containerName="manager"
Aug 04 21:02:57.291816 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.289882    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8ad76b9-d8e9-4fb3-835d-8554e7325ea0" containerName="manager"
Aug 04 21:02:57.291816 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.289956    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8ad76b9-d8e9-4fb3-835d-8554e7325ea0" containerName="manager"
Aug 04 21:02:57.292880 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.292865    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:02:57.297071 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.297050    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n66wc\""
Aug 04 21:02:57.305440 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.305417    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"]
Aug 04 21:02:57.400357 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.400281    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkbph\" (UniqueName: \"kubernetes.io/projected/09cc4962-117c-4ce6-9763-8206b222cd25-kube-api-access-hkbph\") pod \"kuadrant-operator-controller-manager-68cf589589-z2vd2\" (UID: \"09cc4962-117c-4ce6-9763-8206b222cd25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:02:57.400564 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.400405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09cc4962-117c-4ce6-9763-8206b222cd25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-z2vd2\" (UID: \"09cc4962-117c-4ce6-9763-8206b222cd25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:02:57.501407 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.501371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkbph\" (UniqueName: \"kubernetes.io/projected/09cc4962-117c-4ce6-9763-8206b222cd25-kube-api-access-hkbph\") pod \"kuadrant-operator-controller-manager-68cf589589-z2vd2\" (UID: \"09cc4962-117c-4ce6-9763-8206b222cd25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:02:57.501606 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.501418    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09cc4962-117c-4ce6-9763-8206b222cd25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-z2vd2\" (UID: \"09cc4962-117c-4ce6-9763-8206b222cd25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:02:57.501749 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.501732    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09cc4962-117c-4ce6-9763-8206b222cd25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-z2vd2\" (UID: \"09cc4962-117c-4ce6-9763-8206b222cd25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:02:57.513202 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.513178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkbph\" (UniqueName: \"kubernetes.io/projected/09cc4962-117c-4ce6-9763-8206b222cd25-kube-api-access-hkbph\") pod \"kuadrant-operator-controller-manager-68cf589589-z2vd2\" (UID: \"09cc4962-117c-4ce6-9763-8206b222cd25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:02:57.603633 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.603551    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:02:57.748020 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:57.747996    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"]
Aug 04 21:02:57.750760 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:02:57.750730    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09cc4962_117c_4ce6_9763_8206b222cd25.slice/crio-852b6b7b30440c122daa693ebba890ffd9231a242bc4575df0abf257ba69369c WatchSource:0}: Error finding container 852b6b7b30440c122daa693ebba890ffd9231a242bc4575df0abf257ba69369c: Status 404 returned error can't find the container with id 852b6b7b30440c122daa693ebba890ffd9231a242bc4575df0abf257ba69369c
Aug 04 21:02:58.613812 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:58.613779    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2" event={"ID":"09cc4962-117c-4ce6-9763-8206b222cd25","Type":"ContainerStarted","Data":"dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68"}
Aug 04 21:02:58.614399 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:58.613840    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2" event={"ID":"09cc4962-117c-4ce6-9763-8206b222cd25","Type":"ContainerStarted","Data":"852b6b7b30440c122daa693ebba890ffd9231a242bc4575df0abf257ba69369c"}
Aug 04 21:02:58.614399 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:58.613888    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:02:58.635950 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:02:58.635907    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2" podStartSLOduration=1.635888454 podStartE2EDuration="1.635888454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:02:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:02:58.633345767 +0000 UTC m=+1325.876965044" watchObservedRunningTime="2026-08-04 21:02:58.635888454 +0000 UTC m=+1325.879507740"
Aug 04 21:03:09.621589 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:03:09.621556    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:11:17.692543 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.692509    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"]
Aug 04 21:11:17.696118 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.696097    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:17.698518 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.698497    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5a7758b8-serving-cert\""
Aug 04 21:11:17.707608 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.707586    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"]
Aug 04 21:11:17.828000 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.827963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gmfp\" (UniqueName: \"kubernetes.io/projected/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-kube-api-access-6gmfp\") pod \"maas-api-e2e-ait-5a7758b8-858869567-vp7g4\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:17.828181 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.828022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-openshift-service-ca\") pod \"maas-api-e2e-ait-5a7758b8-858869567-vp7g4\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:17.828181 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.828123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-maas-api-tls\") pod \"maas-api-e2e-ait-5a7758b8-858869567-vp7g4\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:17.929458 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.929411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-openshift-service-ca\") pod \"maas-api-e2e-ait-5a7758b8-858869567-vp7g4\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:17.929683 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.929512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-maas-api-tls\") pod \"maas-api-e2e-ait-5a7758b8-858869567-vp7g4\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:17.929683 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.929592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gmfp\" (UniqueName: \"kubernetes.io/projected/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-kube-api-access-6gmfp\") pod \"maas-api-e2e-ait-5a7758b8-858869567-vp7g4\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:17.929683 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:11:17.929660    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-serving-cert: secret "maas-api-e2e-ait-5a7758b8-serving-cert" not found
Aug 04 21:11:17.929869 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:11:17.929735    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-maas-api-tls podName:0c6254ca-132b-4ae0-a2ef-bb11c6213c1c nodeName:}" failed. No retries permitted until 2026-08-04 21:11:18.429713058 +0000 UTC m=+1825.673332328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-maas-api-tls") pod "maas-api-e2e-ait-5a7758b8-858869567-vp7g4" (UID: "0c6254ca-132b-4ae0-a2ef-bb11c6213c1c") : secret "maas-api-e2e-ait-5a7758b8-serving-cert" not found
Aug 04 21:11:17.930228 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.930205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-openshift-service-ca\") pod \"maas-api-e2e-ait-5a7758b8-858869567-vp7g4\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:17.938049 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:17.938024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gmfp\" (UniqueName: \"kubernetes.io/projected/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-kube-api-access-6gmfp\") pod \"maas-api-e2e-ait-5a7758b8-858869567-vp7g4\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:18.434002 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:18.433966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-maas-api-tls\") pod \"maas-api-e2e-ait-5a7758b8-858869567-vp7g4\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:18.436762 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:18.436734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-maas-api-tls\") pod \"maas-api-e2e-ait-5a7758b8-858869567-vp7g4\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:18.608740 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:18.608702    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:18.758532 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:18.758499    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"]
Aug 04 21:11:18.759398 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:11:18.759369    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c6254ca_132b_4ae0_a2ef_bb11c6213c1c.slice/crio-348433eb46ce0125a6924d89e43dffe768b21ec8ac55726ecf843341ebe72121 WatchSource:0}: Error finding container 348433eb46ce0125a6924d89e43dffe768b21ec8ac55726ecf843341ebe72121: Status 404 returned error can't find the container with id 348433eb46ce0125a6924d89e43dffe768b21ec8ac55726ecf843341ebe72121
Aug 04 21:11:18.760848 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:18.760830    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 21:11:19.403131 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:19.403086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4" event={"ID":"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c","Type":"ContainerStarted","Data":"4558a132736faac573d9a86150756b07a9bd9a12970a687df3a6bac58fc1ff4a"}
Aug 04 21:11:19.403310 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:19.403193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4" event={"ID":"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c","Type":"ContainerStarted","Data":"348433eb46ce0125a6924d89e43dffe768b21ec8ac55726ecf843341ebe72121"}
Aug 04 21:11:19.404070 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:19.404049    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:19.421618 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:19.421564    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4" podStartSLOduration=1.917715222 podStartE2EDuration="2.421548365s" totalImagesPullingTime="503.833143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:11:19.420348507 +0000 UTC m=+1826.663967791" watchObservedRunningTime="2026-08-04 21:11:19.421548365 +0000 UTC m=+1826.665167647"
Aug 04 21:11:26.418711 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:26.418682    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:48.404876 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.404838    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"]
Aug 04 21:11:48.405436 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.405101    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4" podUID="0c6254ca-132b-4ae0-a2ef-bb11c6213c1c" containerName="maas-api" containerID="cri-o://4558a132736faac573d9a86150756b07a9bd9a12970a687df3a6bac58fc1ff4a" gracePeriod=30
Aug 04 21:11:48.566995 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.566960    2563 generic.go:356] "Generic (PLEG): container finished" podID="0c6254ca-132b-4ae0-a2ef-bb11c6213c1c" containerID="4558a132736faac573d9a86150756b07a9bd9a12970a687df3a6bac58fc1ff4a" exitCode=0
Aug 04 21:11:48.567157 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.567030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4" event={"ID":"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c","Type":"ContainerDied","Data":"4558a132736faac573d9a86150756b07a9bd9a12970a687df3a6bac58fc1ff4a"}
Aug 04 21:11:48.657053 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.656981    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:48.706341 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.706290    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-openshift-service-ca\") pod \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") "
Aug 04 21:11:48.706526 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.706424    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-maas-api-tls\") pod \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") "
Aug 04 21:11:48.706526 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.706471    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gmfp\" (UniqueName: \"kubernetes.io/projected/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-kube-api-access-6gmfp\") pod \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\" (UID: \"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c\") "
Aug 04 21:11:48.706738 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.706689    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0c6254ca-132b-4ae0-a2ef-bb11c6213c1c" (UID: "0c6254ca-132b-4ae0-a2ef-bb11c6213c1c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:11:48.709437 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.709409    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0c6254ca-132b-4ae0-a2ef-bb11c6213c1c" (UID: "0c6254ca-132b-4ae0-a2ef-bb11c6213c1c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:11:48.709557 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.709511    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-kube-api-access-6gmfp" (OuterVolumeSpecName: "kube-api-access-6gmfp") pod "0c6254ca-132b-4ae0-a2ef-bb11c6213c1c" (UID: "0c6254ca-132b-4ae0-a2ef-bb11c6213c1c"). InnerVolumeSpecName "kube-api-access-6gmfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:11:48.807450 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.807423    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:11:48.807450 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.807449    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gmfp\" (UniqueName: \"kubernetes.io/projected/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-kube-api-access-6gmfp\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:11:48.807628 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:48.807463    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:11:49.573821 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:49.573741    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"
Aug 04 21:11:49.573821 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:49.573774    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4" event={"ID":"0c6254ca-132b-4ae0-a2ef-bb11c6213c1c","Type":"ContainerDied","Data":"348433eb46ce0125a6924d89e43dffe768b21ec8ac55726ecf843341ebe72121"}
Aug 04 21:11:49.574265 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:49.573840    2563 scope.go:117] "RemoveContainer" containerID="4558a132736faac573d9a86150756b07a9bd9a12970a687df3a6bac58fc1ff4a"
Aug 04 21:11:49.595606 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:49.595544    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"]
Aug 04 21:11:49.597975 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:49.597953    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5a7758b8-858869567-vp7g4"]
Aug 04 21:11:51.248610 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:51.248575    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c6254ca-132b-4ae0-a2ef-bb11c6213c1c" path="/var/lib/kubelet/pods/0c6254ca-132b-4ae0-a2ef-bb11c6213c1c/volumes"
Aug 04 21:11:53.145454 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.145413    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"]
Aug 04 21:11:53.146069 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.146048    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c6254ca-132b-4ae0-a2ef-bb11c6213c1c" containerName="maas-api"
Aug 04 21:11:53.146172 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.146072    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c6254ca-132b-4ae0-a2ef-bb11c6213c1c" containerName="maas-api"
Aug 04 21:11:53.146236 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.146180    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c6254ca-132b-4ae0-a2ef-bb11c6213c1c" containerName="maas-api"
Aug 04 21:11:53.155488 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.155457    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:53.158122 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.158070    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-ad77b99e-serving-cert\""
Aug 04 21:11:53.161672 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.161643    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"]
Aug 04 21:11:53.247933 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.247904    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ab1c64-4094-4fa8-a599-73605ed4f250-maas-api-tls\") pod \"maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:53.248114 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.247945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ab1c64-4094-4fa8-a599-73605ed4f250-openshift-service-ca\") pod \"maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:53.248114 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.247966    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg6rk\" (UniqueName: \"kubernetes.io/projected/f5ab1c64-4094-4fa8-a599-73605ed4f250-kube-api-access-xg6rk\") pod \"maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:53.349836 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.349790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ab1c64-4094-4fa8-a599-73605ed4f250-maas-api-tls\") pod \"maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:53.350038 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:11:53.349851    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-serving-cert: secret "maas-api-e2e-migrate-ad77b99e-serving-cert" not found
Aug 04 21:11:53.350038 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.349864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ab1c64-4094-4fa8-a599-73605ed4f250-openshift-service-ca\") pod \"maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:53.350038 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.349893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg6rk\" (UniqueName: \"kubernetes.io/projected/f5ab1c64-4094-4fa8-a599-73605ed4f250-kube-api-access-xg6rk\") pod \"maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:53.350038 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:11:53.349917    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5ab1c64-4094-4fa8-a599-73605ed4f250-maas-api-tls podName:f5ab1c64-4094-4fa8-a599-73605ed4f250 nodeName:}" failed. No retries permitted until 2026-08-04 21:11:53.84989969 +0000 UTC m=+1861.093518953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f5ab1c64-4094-4fa8-a599-73605ed4f250-maas-api-tls") pod "maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz" (UID: "f5ab1c64-4094-4fa8-a599-73605ed4f250") : secret "maas-api-e2e-migrate-ad77b99e-serving-cert" not found
Aug 04 21:11:53.350498 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.350480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ab1c64-4094-4fa8-a599-73605ed4f250-openshift-service-ca\") pod \"maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:53.359746 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.359718    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg6rk\" (UniqueName: \"kubernetes.io/projected/f5ab1c64-4094-4fa8-a599-73605ed4f250-kube-api-access-xg6rk\") pod \"maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:53.854678 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.854641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ab1c64-4094-4fa8-a599-73605ed4f250-maas-api-tls\") pod \"maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:53.857272 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:53.857240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ab1c64-4094-4fa8-a599-73605ed4f250-maas-api-tls\") pod \"maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:54.068170 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:54.068136    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:54.430771 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:54.430744    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"]
Aug 04 21:11:54.432381 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:11:54.432352    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ab1c64_4094_4fa8_a599_73605ed4f250.slice/crio-f711a4be08f3b70506f6d84e0083030b186428855bceda3568d166b126ea38ab WatchSource:0}: Error finding container f711a4be08f3b70506f6d84e0083030b186428855bceda3568d166b126ea38ab: Status 404 returned error can't find the container with id f711a4be08f3b70506f6d84e0083030b186428855bceda3568d166b126ea38ab
Aug 04 21:11:54.603296 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:54.603256    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz" event={"ID":"f5ab1c64-4094-4fa8-a599-73605ed4f250","Type":"ContainerStarted","Data":"f711a4be08f3b70506f6d84e0083030b186428855bceda3568d166b126ea38ab"}
Aug 04 21:11:55.610784 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:55.610741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz" event={"ID":"f5ab1c64-4094-4fa8-a599-73605ed4f250","Type":"ContainerStarted","Data":"cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207"}
Aug 04 21:11:55.611365 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:55.610826    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:11:55.629982 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:11:55.629918    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz" podStartSLOduration=2.2883796849999998 podStartE2EDuration="2.629896175s" totalImagesPullingTime="341.51649ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:11:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:11:55.627969313 +0000 UTC m=+1862.871588596" watchObservedRunningTime="2026-08-04 21:11:55.629896175 +0000 UTC m=+1862.873515459"
Aug 04 21:12:01.622724 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:01.622693    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:12:24.505960 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.505916    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"]
Aug 04 21:12:24.506482 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.506179    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz" podUID="f5ab1c64-4094-4fa8-a599-73605ed4f250" containerName="maas-api" containerID="cri-o://cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207" gracePeriod=30
Aug 04 21:12:24.762876 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.762804    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:12:24.773785 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.773752    2563 generic.go:356] "Generic (PLEG): container finished" podID="f5ab1c64-4094-4fa8-a599-73605ed4f250" containerID="cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207" exitCode=0
Aug 04 21:12:24.773926 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.773899    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz" event={"ID":"f5ab1c64-4094-4fa8-a599-73605ed4f250","Type":"ContainerDied","Data":"cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207"}
Aug 04 21:12:24.773926 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.773911    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"
Aug 04 21:12:24.774065 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.773959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz" event={"ID":"f5ab1c64-4094-4fa8-a599-73605ed4f250","Type":"ContainerDied","Data":"f711a4be08f3b70506f6d84e0083030b186428855bceda3568d166b126ea38ab"}
Aug 04 21:12:24.774065 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.773992    2563 scope.go:117] "RemoveContainer" containerID="cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207"
Aug 04 21:12:24.784492 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.784466    2563 scope.go:117] "RemoveContainer" containerID="cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207"
Aug 04 21:12:24.784817 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:12:24.784796    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207\": container with ID starting with cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207 not found: ID does not exist" containerID="cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207"
Aug 04 21:12:24.784901 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.784832    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207"} err="failed to get container status \"cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207\": rpc error: code = NotFound desc = could not find container \"cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207\": container with ID starting with cc529151a4bbfd8c039d7120228871c83e95c360e7a4f760fdb5c06a2ea04207 not found: ID does not exist"
Aug 04 21:12:24.841635 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.841605    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg6rk\" (UniqueName: \"kubernetes.io/projected/f5ab1c64-4094-4fa8-a599-73605ed4f250-kube-api-access-xg6rk\") pod \"f5ab1c64-4094-4fa8-a599-73605ed4f250\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") "
Aug 04 21:12:24.841635 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.841641    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ab1c64-4094-4fa8-a599-73605ed4f250-maas-api-tls\") pod \"f5ab1c64-4094-4fa8-a599-73605ed4f250\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") "
Aug 04 21:12:24.842054 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.841671    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ab1c64-4094-4fa8-a599-73605ed4f250-openshift-service-ca\") pod \"f5ab1c64-4094-4fa8-a599-73605ed4f250\" (UID: \"f5ab1c64-4094-4fa8-a599-73605ed4f250\") "
Aug 04 21:12:24.842054 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.841991    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ab1c64-4094-4fa8-a599-73605ed4f250-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f5ab1c64-4094-4fa8-a599-73605ed4f250" (UID: "f5ab1c64-4094-4fa8-a599-73605ed4f250"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:12:24.844436 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.844399    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5ab1c64-4094-4fa8-a599-73605ed4f250-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f5ab1c64-4094-4fa8-a599-73605ed4f250" (UID: "f5ab1c64-4094-4fa8-a599-73605ed4f250"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:12:24.844554 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.844504    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5ab1c64-4094-4fa8-a599-73605ed4f250-kube-api-access-xg6rk" (OuterVolumeSpecName: "kube-api-access-xg6rk") pod "f5ab1c64-4094-4fa8-a599-73605ed4f250" (UID: "f5ab1c64-4094-4fa8-a599-73605ed4f250"). InnerVolumeSpecName "kube-api-access-xg6rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:12:24.943174 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.943134    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg6rk\" (UniqueName: \"kubernetes.io/projected/f5ab1c64-4094-4fa8-a599-73605ed4f250-kube-api-access-xg6rk\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:12:24.943174 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.943170    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ab1c64-4094-4fa8-a599-73605ed4f250-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:12:24.943174 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:24.943180    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ab1c64-4094-4fa8-a599-73605ed4f250-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:12:25.098792 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:25.098761    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"]
Aug 04 21:12:25.102065 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:25.102041    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad77b99e-6dfbd6c687-xm4mz"]
Aug 04 21:12:25.243942 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:25.243908    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5ab1c64-4094-4fa8-a599-73605ed4f250" path="/var/lib/kubelet/pods/f5ab1c64-4094-4fa8-a599-73605ed4f250/volumes"
Aug 04 21:12:29.152797 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.152754    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"]
Aug 04 21:12:29.153402 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.153376    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5ab1c64-4094-4fa8-a599-73605ed4f250" containerName="maas-api"
Aug 04 21:12:29.153402 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.153403    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5ab1c64-4094-4fa8-a599-73605ed4f250" containerName="maas-api"
Aug 04 21:12:29.153558 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.153528    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5ab1c64-4094-4fa8-a599-73605ed4f250" containerName="maas-api"
Aug 04 21:12:29.155628 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.155609    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:29.157980 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.157958    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-213b68bd-serving-cert\""
Aug 04 21:12:29.165738 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.165715    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"]
Aug 04 21:12:29.179388 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.179335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a9132f-5e23-479a-8bc5-603e2d7ec745-maas-api-tls\") pod \"maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:29.179575 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.179403    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrgsm\" (UniqueName: \"kubernetes.io/projected/54a9132f-5e23-479a-8bc5-603e2d7ec745-kube-api-access-jrgsm\") pod \"maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:29.179575 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.179498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a9132f-5e23-479a-8bc5-603e2d7ec745-openshift-service-ca\") pod \"maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:29.280885 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.280842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a9132f-5e23-479a-8bc5-603e2d7ec745-openshift-service-ca\") pod \"maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:29.281088 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.280997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a9132f-5e23-479a-8bc5-603e2d7ec745-maas-api-tls\") pod \"maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:29.281088 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.281056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrgsm\" (UniqueName: \"kubernetes.io/projected/54a9132f-5e23-479a-8bc5-603e2d7ec745-kube-api-access-jrgsm\") pod \"maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:29.281280 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:12:29.281148    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-serving-cert: secret "maas-api-e2e-ait-213b68bd-serving-cert" not found
Aug 04 21:12:29.281388 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:12:29.281336    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a9132f-5e23-479a-8bc5-603e2d7ec745-maas-api-tls podName:54a9132f-5e23-479a-8bc5-603e2d7ec745 nodeName:}" failed. No retries permitted until 2026-08-04 21:12:29.781291435 +0000 UTC m=+1897.024910718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/54a9132f-5e23-479a-8bc5-603e2d7ec745-maas-api-tls") pod "maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7" (UID: "54a9132f-5e23-479a-8bc5-603e2d7ec745") : secret "maas-api-e2e-ait-213b68bd-serving-cert" not found
Aug 04 21:12:29.281639 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.281618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a9132f-5e23-479a-8bc5-603e2d7ec745-openshift-service-ca\") pod \"maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:29.291666 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.291642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrgsm\" (UniqueName: \"kubernetes.io/projected/54a9132f-5e23-479a-8bc5-603e2d7ec745-kube-api-access-jrgsm\") pod \"maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:29.786980 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.786947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a9132f-5e23-479a-8bc5-603e2d7ec745-maas-api-tls\") pod \"maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:29.789624 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:29.789595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a9132f-5e23-479a-8bc5-603e2d7ec745-maas-api-tls\") pod \"maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:30.067531 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:30.067442    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:30.220856 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:30.220823    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"]
Aug 04 21:12:30.223187 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:12:30.223156    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a9132f_5e23_479a_8bc5_603e2d7ec745.slice/crio-2976678e041ec4d7fd759361c4215d5b86569c7c7a8918a9c25736599a2dfbdd WatchSource:0}: Error finding container 2976678e041ec4d7fd759361c4215d5b86569c7c7a8918a9c25736599a2dfbdd: Status 404 returned error can't find the container with id 2976678e041ec4d7fd759361c4215d5b86569c7c7a8918a9c25736599a2dfbdd
Aug 04 21:12:30.809737 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:30.809702    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7" event={"ID":"54a9132f-5e23-479a-8bc5-603e2d7ec745","Type":"ContainerStarted","Data":"25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147"}
Aug 04 21:12:30.809927 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:30.809762    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7" event={"ID":"54a9132f-5e23-479a-8bc5-603e2d7ec745","Type":"ContainerStarted","Data":"2976678e041ec4d7fd759361c4215d5b86569c7c7a8918a9c25736599a2dfbdd"}
Aug 04 21:12:30.809927 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:30.809791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:30.830006 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:30.829903    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7" podStartSLOduration=1.480801183 podStartE2EDuration="1.829881435s" totalImagesPullingTime="349.080252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:12:30.826908749 +0000 UTC m=+1898.070528032" watchObservedRunningTime="2026-08-04 21:12:30.829881435 +0000 UTC m=+1898.073500719"
Aug 04 21:12:36.821862 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:36.821829    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:59.644109 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:59.644061    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"]
Aug 04 21:12:59.644695 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:59.644356    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7" podUID="54a9132f-5e23-479a-8bc5-603e2d7ec745" containerName="maas-api" containerID="cri-o://25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147" gracePeriod=30
Aug 04 21:12:59.902526 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:59.902459    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:59.987143 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:59.987109    2563 generic.go:356] "Generic (PLEG): container finished" podID="54a9132f-5e23-479a-8bc5-603e2d7ec745" containerID="25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147" exitCode=0
Aug 04 21:12:59.987307 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:59.987150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7" event={"ID":"54a9132f-5e23-479a-8bc5-603e2d7ec745","Type":"ContainerDied","Data":"25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147"}
Aug 04 21:12:59.987307 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:59.987205    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7" event={"ID":"54a9132f-5e23-479a-8bc5-603e2d7ec745","Type":"ContainerDied","Data":"2976678e041ec4d7fd759361c4215d5b86569c7c7a8918a9c25736599a2dfbdd"}
Aug 04 21:12:59.987307 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:59.987159    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"
Aug 04 21:12:59.987307 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:59.987229    2563 scope.go:117] "RemoveContainer" containerID="25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147"
Aug 04 21:12:59.999568 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:59.999535    2563 scope.go:117] "RemoveContainer" containerID="25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147"
Aug 04 21:12:59.999960 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:12:59.999932    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147\": container with ID starting with 25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147 not found: ID does not exist" containerID="25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147"
Aug 04 21:13:00.000054 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:12:59.999966    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147"} err="failed to get container status \"25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147\": rpc error: code = NotFound desc = could not find container \"25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147\": container with ID starting with 25d5dadeba7919d7a68978f2af5d8b9a1cd590417c6e860d28cabb6da6aca147 not found: ID does not exist"
Aug 04 21:13:00.064170 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:00.064138    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a9132f-5e23-479a-8bc5-603e2d7ec745-openshift-service-ca\") pod \"54a9132f-5e23-479a-8bc5-603e2d7ec745\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") "
Aug 04 21:13:00.064420 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:00.064195    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a9132f-5e23-479a-8bc5-603e2d7ec745-maas-api-tls\") pod \"54a9132f-5e23-479a-8bc5-603e2d7ec745\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") "
Aug 04 21:13:00.064420 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:00.064248    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrgsm\" (UniqueName: \"kubernetes.io/projected/54a9132f-5e23-479a-8bc5-603e2d7ec745-kube-api-access-jrgsm\") pod \"54a9132f-5e23-479a-8bc5-603e2d7ec745\" (UID: \"54a9132f-5e23-479a-8bc5-603e2d7ec745\") "
Aug 04 21:13:00.064626 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:00.064601    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54a9132f-5e23-479a-8bc5-603e2d7ec745-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "54a9132f-5e23-479a-8bc5-603e2d7ec745" (UID: "54a9132f-5e23-479a-8bc5-603e2d7ec745"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:13:00.066491 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:00.066463    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54a9132f-5e23-479a-8bc5-603e2d7ec745-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "54a9132f-5e23-479a-8bc5-603e2d7ec745" (UID: "54a9132f-5e23-479a-8bc5-603e2d7ec745"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:13:00.066583 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:00.066514    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54a9132f-5e23-479a-8bc5-603e2d7ec745-kube-api-access-jrgsm" (OuterVolumeSpecName: "kube-api-access-jrgsm") pod "54a9132f-5e23-479a-8bc5-603e2d7ec745" (UID: "54a9132f-5e23-479a-8bc5-603e2d7ec745"). InnerVolumeSpecName "kube-api-access-jrgsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:13:00.165082 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:00.165005    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a9132f-5e23-479a-8bc5-603e2d7ec745-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:13:00.165082 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:00.165032    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrgsm\" (UniqueName: \"kubernetes.io/projected/54a9132f-5e23-479a-8bc5-603e2d7ec745-kube-api-access-jrgsm\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:13:00.165082 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:00.165042    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a9132f-5e23-479a-8bc5-603e2d7ec745-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:13:00.311350 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:00.311309    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"]
Aug 04 21:13:00.315087 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:00.315058    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-213b68bd-756dc94cbd-g28j7"]
Aug 04 21:13:01.243180 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:01.243146    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54a9132f-5e23-479a-8bc5-603e2d7ec745" path="/var/lib/kubelet/pods/54a9132f-5e23-479a-8bc5-603e2d7ec745/volumes"
Aug 04 21:13:05.461657 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.461620    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"]
Aug 04 21:13:05.462036 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.462020    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54a9132f-5e23-479a-8bc5-603e2d7ec745" containerName="maas-api"
Aug 04 21:13:05.462036 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.462031    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="54a9132f-5e23-479a-8bc5-603e2d7ec745" containerName="maas-api"
Aug 04 21:13:05.462143 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.462090    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="54a9132f-5e23-479a-8bc5-603e2d7ec745" containerName="maas-api"
Aug 04 21:13:05.466593 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.466572    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:05.468954 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.468935    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-043d8f1f-serving-cert\""
Aug 04 21:13:05.474918 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.474892    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"]
Aug 04 21:13:05.514402 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.514362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-openshift-service-ca\") pod \"maas-api-e2e-derive-043d8f1f-6999d74ff-lffln\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:05.514577 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.514460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngm97\" (UniqueName: \"kubernetes.io/projected/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-kube-api-access-ngm97\") pod \"maas-api-e2e-derive-043d8f1f-6999d74ff-lffln\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:05.514577 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.514493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-maas-api-tls\") pod \"maas-api-e2e-derive-043d8f1f-6999d74ff-lffln\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:05.615565 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.615526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-maas-api-tls\") pod \"maas-api-e2e-derive-043d8f1f-6999d74ff-lffln\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:05.615741 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.615598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-openshift-service-ca\") pod \"maas-api-e2e-derive-043d8f1f-6999d74ff-lffln\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:05.615741 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.615640    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngm97\" (UniqueName: \"kubernetes.io/projected/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-kube-api-access-ngm97\") pod \"maas-api-e2e-derive-043d8f1f-6999d74ff-lffln\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:05.615741 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:13:05.615688    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-serving-cert: secret "maas-api-e2e-derive-043d8f1f-serving-cert" not found
Aug 04 21:13:05.615851 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:13:05.615757    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-maas-api-tls podName:6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43 nodeName:}" failed. No retries permitted until 2026-08-04 21:13:06.115734627 +0000 UTC m=+1933.359353891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-maas-api-tls") pod "maas-api-e2e-derive-043d8f1f-6999d74ff-lffln" (UID: "6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43") : secret "maas-api-e2e-derive-043d8f1f-serving-cert" not found
Aug 04 21:13:05.616253 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.616233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-openshift-service-ca\") pod \"maas-api-e2e-derive-043d8f1f-6999d74ff-lffln\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:05.625669 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:05.625643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngm97\" (UniqueName: \"kubernetes.io/projected/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-kube-api-access-ngm97\") pod \"maas-api-e2e-derive-043d8f1f-6999d74ff-lffln\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:06.119427 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:06.119383    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-maas-api-tls\") pod \"maas-api-e2e-derive-043d8f1f-6999d74ff-lffln\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:06.122073 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:06.122043    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-maas-api-tls\") pod \"maas-api-e2e-derive-043d8f1f-6999d74ff-lffln\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:06.379077 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:06.378973    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:06.533024 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:06.532996    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"]
Aug 04 21:13:06.534792 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:13:06.534769    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc34618_33b6_4fd8_9b8f_8cc5aa92cd43.slice/crio-e95e364b991f70cb643044b89c5e6f66bad242c5f3b36550964c952ce00f5d90 WatchSource:0}: Error finding container e95e364b991f70cb643044b89c5e6f66bad242c5f3b36550964c952ce00f5d90: Status 404 returned error can't find the container with id e95e364b991f70cb643044b89c5e6f66bad242c5f3b36550964c952ce00f5d90
Aug 04 21:13:07.028897 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:07.028859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln" event={"ID":"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43","Type":"ContainerStarted","Data":"d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8"}
Aug 04 21:13:07.029092 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:07.028925    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln" event={"ID":"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43","Type":"ContainerStarted","Data":"e95e364b991f70cb643044b89c5e6f66bad242c5f3b36550964c952ce00f5d90"}
Aug 04 21:13:07.029092 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:07.029048    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:07.047851 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:07.047804    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln" podStartSLOduration=1.689243382 podStartE2EDuration="2.047790299s" totalImagesPullingTime="358.546917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:13:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:13:07.045468342 +0000 UTC m=+1934.289087650" watchObservedRunningTime="2026-08-04 21:13:07.047790299 +0000 UTC m=+1934.291409636"
Aug 04 21:13:13.041158 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:13.041131    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:36.080130 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.080091    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"]
Aug 04 21:13:36.080762 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.080399    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln" podUID="6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43" containerName="maas-api" containerID="cri-o://d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8" gracePeriod=30
Aug 04 21:13:36.336695 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.336635    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:36.499286 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.499253    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngm97\" (UniqueName: \"kubernetes.io/projected/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-kube-api-access-ngm97\") pod \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") "
Aug 04 21:13:36.499472 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.499337    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-maas-api-tls\") pod \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") "
Aug 04 21:13:36.499472 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.499361    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-openshift-service-ca\") pod \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\" (UID: \"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43\") "
Aug 04 21:13:36.499842 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.499803    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43" (UID: "6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:13:36.501580 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.501550    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43" (UID: "6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:13:36.501696 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.501582    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-kube-api-access-ngm97" (OuterVolumeSpecName: "kube-api-access-ngm97") pod "6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43" (UID: "6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43"). InnerVolumeSpecName "kube-api-access-ngm97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:13:36.600212 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.600134    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:13:36.600212 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.600163    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:13:36.600212 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:36.600174    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngm97\" (UniqueName: \"kubernetes.io/projected/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43-kube-api-access-ngm97\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:13:37.199553 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:37.199518    2563 generic.go:356] "Generic (PLEG): container finished" podID="6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43" containerID="d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8" exitCode=0
Aug 04 21:13:37.199991 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:37.199573    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"
Aug 04 21:13:37.199991 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:37.199580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln" event={"ID":"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43","Type":"ContainerDied","Data":"d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8"}
Aug 04 21:13:37.199991 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:37.199630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln" event={"ID":"6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43","Type":"ContainerDied","Data":"e95e364b991f70cb643044b89c5e6f66bad242c5f3b36550964c952ce00f5d90"}
Aug 04 21:13:37.199991 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:37.199653    2563 scope.go:117] "RemoveContainer" containerID="d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8"
Aug 04 21:13:37.209463 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:37.209442    2563 scope.go:117] "RemoveContainer" containerID="d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8"
Aug 04 21:13:37.209711 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:13:37.209693    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8\": container with ID starting with d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8 not found: ID does not exist" containerID="d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8"
Aug 04 21:13:37.209762 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:37.209720    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8"} err="failed to get container status \"d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8\": rpc error: code = NotFound desc = could not find container \"d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8\": container with ID starting with d01bc3552d572d27ed80fedc9ca69ae59142e857f14218d72656404df251e5e8 not found: ID does not exist"
Aug 04 21:13:37.229114 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:37.229088    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"]
Aug 04 21:13:37.231594 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:37.231574    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-043d8f1f-6999d74ff-lffln"]
Aug 04 21:13:37.243656 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:13:37.243630    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43" path="/var/lib/kubelet/pods/6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43/volumes"
Aug 04 21:15:40.576832 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.576795    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"]
Aug 04 21:15:40.577256 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.577177    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43" containerName="maas-api"
Aug 04 21:15:40.577256 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.577188    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43" containerName="maas-api"
Aug 04 21:15:40.577256 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.577247    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fc34618-33b6-4fd8-9b8f-8cc5aa92cd43" containerName="maas-api"
Aug 04 21:15:40.580384 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.580353    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:40.583018 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.582998    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2fcd204a-serving-cert\""
Aug 04 21:15:40.589522 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.589495    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"]
Aug 04 21:15:40.642914 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.642881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-maas-api-tls\") pod \"maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:40.642914 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.642923    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fb9h\" (UniqueName: \"kubernetes.io/projected/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-kube-api-access-2fb9h\") pod \"maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:40.643166 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.642970    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-openshift-service-ca\") pod \"maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:40.744383 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.744304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-maas-api-tls\") pod \"maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:40.744609 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.744397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fb9h\" (UniqueName: \"kubernetes.io/projected/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-kube-api-access-2fb9h\") pod \"maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:40.744609 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.744435    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-openshift-service-ca\") pod \"maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:40.744609 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:15:40.744457    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-serving-cert: secret "maas-api-e2e-mt-2fcd204a-serving-cert" not found
Aug 04 21:15:40.744609 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:15:40.744527    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-maas-api-tls podName:dcd3b66b-7523-406e-8aad-ebd86fda1e6b nodeName:}" failed. No retries permitted until 2026-08-04 21:15:41.244511253 +0000 UTC m=+2088.488130513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-maas-api-tls") pod "maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz" (UID: "dcd3b66b-7523-406e-8aad-ebd86fda1e6b") : secret "maas-api-e2e-mt-2fcd204a-serving-cert" not found
Aug 04 21:15:40.745145 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.745124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-openshift-service-ca\") pod \"maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:40.753371 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:40.753344    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fb9h\" (UniqueName: \"kubernetes.io/projected/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-kube-api-access-2fb9h\") pod \"maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:41.249431 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:41.249397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-maas-api-tls\") pod \"maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:41.252001 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:41.251971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-maas-api-tls\") pod \"maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:41.492893 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:41.492857    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:41.649727 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:41.649698    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"]
Aug 04 21:15:41.651930 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:15:41.651906    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd3b66b_7523_406e_8aad_ebd86fda1e6b.slice/crio-ef57a3ae70dd2b19997050a48027c2fcbcf40c6103816aaf72db5dd8ff44d7fb WatchSource:0}: Error finding container ef57a3ae70dd2b19997050a48027c2fcbcf40c6103816aaf72db5dd8ff44d7fb: Status 404 returned error can't find the container with id ef57a3ae70dd2b19997050a48027c2fcbcf40c6103816aaf72db5dd8ff44d7fb
Aug 04 21:15:41.873378 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:41.873350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz" event={"ID":"dcd3b66b-7523-406e-8aad-ebd86fda1e6b","Type":"ContainerStarted","Data":"ef57a3ae70dd2b19997050a48027c2fcbcf40c6103816aaf72db5dd8ff44d7fb"}
Aug 04 21:15:42.880607 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:42.880568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz" event={"ID":"dcd3b66b-7523-406e-8aad-ebd86fda1e6b","Type":"ContainerStarted","Data":"d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f"}
Aug 04 21:15:42.881033 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:42.880777    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:15:42.900155 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:42.900106    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz" podStartSLOduration=2.468053692 podStartE2EDuration="2.900090671s" totalImagesPullingTime="432.036979ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:15:42.898540311 +0000 UTC m=+2090.142159593" watchObservedRunningTime="2026-08-04 21:15:42.900090671 +0000 UTC m=+2090.143709952"
Aug 04 21:15:48.891867 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:15:48.891835    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:16:22.265859 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.265818    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"]
Aug 04 21:16:22.269447 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.266147    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz" podUID="dcd3b66b-7523-406e-8aad-ebd86fda1e6b" containerName="maas-api" containerID="cri-o://d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f" gracePeriod=30
Aug 04 21:16:22.522649 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.522589    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:16:22.622835 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.622799    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-openshift-service-ca\") pod \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") "
Aug 04 21:16:22.623006 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.622855    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-maas-api-tls\") pod \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") "
Aug 04 21:16:22.623006 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.622955    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fb9h\" (UniqueName: \"kubernetes.io/projected/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-kube-api-access-2fb9h\") pod \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\" (UID: \"dcd3b66b-7523-406e-8aad-ebd86fda1e6b\") "
Aug 04 21:16:22.623282 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.623252    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dcd3b66b-7523-406e-8aad-ebd86fda1e6b" (UID: "dcd3b66b-7523-406e-8aad-ebd86fda1e6b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:16:22.625157 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.625129    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dcd3b66b-7523-406e-8aad-ebd86fda1e6b" (UID: "dcd3b66b-7523-406e-8aad-ebd86fda1e6b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:16:22.625270 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.625207    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-kube-api-access-2fb9h" (OuterVolumeSpecName: "kube-api-access-2fb9h") pod "dcd3b66b-7523-406e-8aad-ebd86fda1e6b" (UID: "dcd3b66b-7523-406e-8aad-ebd86fda1e6b"). InnerVolumeSpecName "kube-api-access-2fb9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:16:22.723839 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.723798    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fb9h\" (UniqueName: \"kubernetes.io/projected/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-kube-api-access-2fb9h\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:16:22.723839 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.723832    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:16:22.723839 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:22.723844    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd3b66b-7523-406e-8aad-ebd86fda1e6b-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:16:23.097973 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:23.097938    2563 generic.go:356] "Generic (PLEG): container finished" podID="dcd3b66b-7523-406e-8aad-ebd86fda1e6b" containerID="d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f" exitCode=0
Aug 04 21:16:23.098178 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:23.097983    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"
Aug 04 21:16:23.098178 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:23.098026    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz" event={"ID":"dcd3b66b-7523-406e-8aad-ebd86fda1e6b","Type":"ContainerDied","Data":"d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f"}
Aug 04 21:16:23.098178 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:23.098080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz" event={"ID":"dcd3b66b-7523-406e-8aad-ebd86fda1e6b","Type":"ContainerDied","Data":"ef57a3ae70dd2b19997050a48027c2fcbcf40c6103816aaf72db5dd8ff44d7fb"}
Aug 04 21:16:23.098178 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:23.098104    2563 scope.go:117] "RemoveContainer" containerID="d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f"
Aug 04 21:16:23.108792 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:23.108769    2563 scope.go:117] "RemoveContainer" containerID="d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f"
Aug 04 21:16:23.109091 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:16:23.109070    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f\": container with ID starting with d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f not found: ID does not exist" containerID="d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f"
Aug 04 21:16:23.109160 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:23.109106    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f"} err="failed to get container status \"d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f\": rpc error: code = NotFound desc = could not find container \"d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f\": container with ID starting with d7650a307c3feba1f4b2d31ed4d3e985ed8573cac9f50fcbec12453ceb96b44f not found: ID does not exist"
Aug 04 21:16:23.119772 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:23.119746    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"]
Aug 04 21:16:23.123494 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:23.123472    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fcd204a-7795b4fcb5-mjsfz"]
Aug 04 21:16:23.242940 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:16:23.242910    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcd3b66b-7523-406e-8aad-ebd86fda1e6b" path="/var/lib/kubelet/pods/dcd3b66b-7523-406e-8aad-ebd86fda1e6b/volumes"
Aug 04 21:17:10.666434 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:10.666396    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"]
Aug 04 21:17:10.666927 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:10.666688    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2" podUID="09cc4962-117c-4ce6-9763-8206b222cd25" containerName="manager" containerID="cri-o://dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68" gracePeriod=10
Aug 04 21:17:10.927654 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:10.927585    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:17:10.937998 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:10.937965    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkbph\" (UniqueName: \"kubernetes.io/projected/09cc4962-117c-4ce6-9763-8206b222cd25-kube-api-access-hkbph\") pod \"09cc4962-117c-4ce6-9763-8206b222cd25\" (UID: \"09cc4962-117c-4ce6-9763-8206b222cd25\") "
Aug 04 21:17:10.938170 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:10.938031    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09cc4962-117c-4ce6-9763-8206b222cd25-extensions-socket-volume\") pod \"09cc4962-117c-4ce6-9763-8206b222cd25\" (UID: \"09cc4962-117c-4ce6-9763-8206b222cd25\") "
Aug 04 21:17:10.938420 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:10.938395    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09cc4962-117c-4ce6-9763-8206b222cd25-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "09cc4962-117c-4ce6-9763-8206b222cd25" (UID: "09cc4962-117c-4ce6-9763-8206b222cd25"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 21:17:10.940384 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:10.940354    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09cc4962-117c-4ce6-9763-8206b222cd25-kube-api-access-hkbph" (OuterVolumeSpecName: "kube-api-access-hkbph") pod "09cc4962-117c-4ce6-9763-8206b222cd25" (UID: "09cc4962-117c-4ce6-9763-8206b222cd25"). InnerVolumeSpecName "kube-api-access-hkbph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:17:11.039101 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:11.039066    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkbph\" (UniqueName: \"kubernetes.io/projected/09cc4962-117c-4ce6-9763-8206b222cd25-kube-api-access-hkbph\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:17:11.039101 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:11.039098    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09cc4962-117c-4ce6-9763-8206b222cd25-extensions-socket-volume\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:17:11.361233 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:11.361200    2563 generic.go:356] "Generic (PLEG): container finished" podID="09cc4962-117c-4ce6-9763-8206b222cd25" containerID="dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68" exitCode=0
Aug 04 21:17:11.361454 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:11.361247    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"
Aug 04 21:17:11.361454 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:11.361271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2" event={"ID":"09cc4962-117c-4ce6-9763-8206b222cd25","Type":"ContainerDied","Data":"dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68"}
Aug 04 21:17:11.361454 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:11.361340    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2" event={"ID":"09cc4962-117c-4ce6-9763-8206b222cd25","Type":"ContainerDied","Data":"852b6b7b30440c122daa693ebba890ffd9231a242bc4575df0abf257ba69369c"}
Aug 04 21:17:11.361454 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:11.361369    2563 scope.go:117] "RemoveContainer" containerID="dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68"
Aug 04 21:17:11.371580 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:11.371555    2563 scope.go:117] "RemoveContainer" containerID="dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68"
Aug 04 21:17:11.371862 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:17:11.371843    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68\": container with ID starting with dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68 not found: ID does not exist" containerID="dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68"
Aug 04 21:17:11.371937 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:11.371873    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68"} err="failed to get container status \"dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68\": rpc error: code = NotFound desc = could not find container \"dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68\": container with ID starting with dd430b892dbb3fde4fdca3998334eee984315e0bf574cd0f96da9d97a6b02f68 not found: ID does not exist"
Aug 04 21:17:11.385774 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:11.385752    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"]
Aug 04 21:17:11.389915 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:11.389895    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-z2vd2"]
Aug 04 21:17:13.243824 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:17:13.243791    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09cc4962-117c-4ce6-9763-8206b222cd25" path="/var/lib/kubelet/pods/09cc4962-117c-4ce6-9763-8206b222cd25/volumes"
Aug 04 21:18:26.309955 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.309866    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"]
Aug 04 21:18:26.310468 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.310269    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09cc4962-117c-4ce6-9763-8206b222cd25" containerName="manager"
Aug 04 21:18:26.310468 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.310281    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="09cc4962-117c-4ce6-9763-8206b222cd25" containerName="manager"
Aug 04 21:18:26.310468 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.310290    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd3b66b-7523-406e-8aad-ebd86fda1e6b" containerName="maas-api"
Aug 04 21:18:26.310468 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.310296    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd3b66b-7523-406e-8aad-ebd86fda1e6b" containerName="maas-api"
Aug 04 21:18:26.310468 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.310385    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="09cc4962-117c-4ce6-9763-8206b222cd25" containerName="manager"
Aug 04 21:18:26.310468 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.310397    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcd3b66b-7523-406e-8aad-ebd86fda1e6b" containerName="maas-api"
Aug 04 21:18:26.313570 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.313552    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"
Aug 04 21:18:26.316201 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.316176    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n66wc\""
Aug 04 21:18:26.342810 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.342769    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"]
Aug 04 21:18:26.389415 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.389370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgh4n\" (UniqueName: \"kubernetes.io/projected/72f0c04c-30f8-4f74-9848-142de69eca43-kube-api-access-dgh4n\") pod \"kuadrant-operator-controller-manager-68cf589589-q4fjn\" (UID: \"72f0c04c-30f8-4f74-9848-142de69eca43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"
Aug 04 21:18:26.389611 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.389425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72f0c04c-30f8-4f74-9848-142de69eca43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-q4fjn\" (UID: \"72f0c04c-30f8-4f74-9848-142de69eca43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"
Aug 04 21:18:26.490842 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.490803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgh4n\" (UniqueName: \"kubernetes.io/projected/72f0c04c-30f8-4f74-9848-142de69eca43-kube-api-access-dgh4n\") pod \"kuadrant-operator-controller-manager-68cf589589-q4fjn\" (UID: \"72f0c04c-30f8-4f74-9848-142de69eca43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"
Aug 04 21:18:26.490842 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.490846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72f0c04c-30f8-4f74-9848-142de69eca43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-q4fjn\" (UID: \"72f0c04c-30f8-4f74-9848-142de69eca43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"
Aug 04 21:18:26.491245 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.491226    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/72f0c04c-30f8-4f74-9848-142de69eca43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-q4fjn\" (UID: \"72f0c04c-30f8-4f74-9848-142de69eca43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"
Aug 04 21:18:26.500887 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.500855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgh4n\" (UniqueName: \"kubernetes.io/projected/72f0c04c-30f8-4f74-9848-142de69eca43-kube-api-access-dgh4n\") pod \"kuadrant-operator-controller-manager-68cf589589-q4fjn\" (UID: \"72f0c04c-30f8-4f74-9848-142de69eca43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"
Aug 04 21:18:26.624097 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.624072    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"
Aug 04 21:18:26.777189 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:26.777158    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"]
Aug 04 21:18:26.778484 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:18:26.778460    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f0c04c_30f8_4f74_9848_142de69eca43.slice/crio-820eb3ccbbb5dabe43f272ed0b4aa9567d1315e963a3418981e9a0571077d90b WatchSource:0}: Error finding container 820eb3ccbbb5dabe43f272ed0b4aa9567d1315e963a3418981e9a0571077d90b: Status 404 returned error can't find the container with id 820eb3ccbbb5dabe43f272ed0b4aa9567d1315e963a3418981e9a0571077d90b
Aug 04 21:18:27.770045 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:27.770013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn" event={"ID":"72f0c04c-30f8-4f74-9848-142de69eca43","Type":"ContainerStarted","Data":"1ac92dde75c9da955f192995a429cdf67cf8760115e5d7b7736a30d671585054"}
Aug 04 21:18:27.770460 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:27.770072    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn" event={"ID":"72f0c04c-30f8-4f74-9848-142de69eca43","Type":"ContainerStarted","Data":"820eb3ccbbb5dabe43f272ed0b4aa9567d1315e963a3418981e9a0571077d90b"}
Aug 04 21:18:27.770460 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:27.770150    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"
Aug 04 21:18:27.796862 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:27.796812    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn" podStartSLOduration=1.796796558 podStartE2EDuration="1.796796558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:18:27.793642013 +0000 UTC m=+2255.037261312" watchObservedRunningTime="2026-08-04 21:18:27.796796558 +0000 UTC m=+2255.040415841"
Aug 04 21:18:38.778433 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:38.778403    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q4fjn"
Aug 04 21:18:49.551550 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.551508    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"]
Aug 04 21:18:49.555876 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.555854    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:49.558547 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.558519    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d1f5ec52-serving-cert\""
Aug 04 21:18:49.571829 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.571798    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"]
Aug 04 21:18:49.704227 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.704181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmtbd\" (UniqueName: \"kubernetes.io/projected/66dae5ff-f41d-496a-bd91-d11fcee34b83-kube-api-access-kmtbd\") pod \"maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:49.704461 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.704268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66dae5ff-f41d-496a-bd91-d11fcee34b83-maas-api-tls\") pod \"maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:49.704461 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.704308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66dae5ff-f41d-496a-bd91-d11fcee34b83-openshift-service-ca\") pod \"maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:49.805362 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.805233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66dae5ff-f41d-496a-bd91-d11fcee34b83-maas-api-tls\") pod \"maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:49.805362 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.805304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66dae5ff-f41d-496a-bd91-d11fcee34b83-openshift-service-ca\") pod \"maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:49.805614 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.805402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmtbd\" (UniqueName: \"kubernetes.io/projected/66dae5ff-f41d-496a-bd91-d11fcee34b83-kube-api-access-kmtbd\") pod \"maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:49.805614 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:18:49.805435    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-serving-cert: secret "maas-api-e2e-mt-d1f5ec52-serving-cert" not found
Aug 04 21:18:49.805614 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:18:49.805520    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66dae5ff-f41d-496a-bd91-d11fcee34b83-maas-api-tls podName:66dae5ff-f41d-496a-bd91-d11fcee34b83 nodeName:}" failed. No retries permitted until 2026-08-04 21:18:50.305487994 +0000 UTC m=+2277.549107254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/66dae5ff-f41d-496a-bd91-d11fcee34b83-maas-api-tls") pod "maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh" (UID: "66dae5ff-f41d-496a-bd91-d11fcee34b83") : secret "maas-api-e2e-mt-d1f5ec52-serving-cert" not found
Aug 04 21:18:49.806122 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.806095    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66dae5ff-f41d-496a-bd91-d11fcee34b83-openshift-service-ca\") pod \"maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:49.813891 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:49.813859    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmtbd\" (UniqueName: \"kubernetes.io/projected/66dae5ff-f41d-496a-bd91-d11fcee34b83-kube-api-access-kmtbd\") pod \"maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:50.311851 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:50.311814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66dae5ff-f41d-496a-bd91-d11fcee34b83-maas-api-tls\") pod \"maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:50.314492 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:50.314460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66dae5ff-f41d-496a-bd91-d11fcee34b83-maas-api-tls\") pod \"maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:50.467877 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:50.467841    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:50.821152 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:50.821124    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"]
Aug 04 21:18:50.826189 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:50.826168    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 21:18:50.897589 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:50.897550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh" event={"ID":"66dae5ff-f41d-496a-bd91-d11fcee34b83","Type":"ContainerStarted","Data":"b4797c0a3fc3829a9a7fb8d0ff19a2381a4d5c38c50865eea379c981153d9d79"}
Aug 04 21:18:51.904955 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:51.904906    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh" event={"ID":"66dae5ff-f41d-496a-bd91-d11fcee34b83","Type":"ContainerStarted","Data":"a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f"}
Aug 04 21:18:51.905609 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:51.904987    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:18:51.925257 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:51.925203    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh" podStartSLOduration=2.455447952 podStartE2EDuration="2.925185967s" totalImagesPullingTime="469.738015ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:18:51.923153752 +0000 UTC m=+2279.166773037" watchObservedRunningTime="2026-08-04 21:18:51.925185967 +0000 UTC m=+2279.168805250"
Aug 04 21:18:57.916103 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:18:57.916076    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:20:03.250597 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.250511    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"]
Aug 04 21:20:03.251108 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.250784    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh" podUID="66dae5ff-f41d-496a-bd91-d11fcee34b83" containerName="maas-api" containerID="cri-o://a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f" gracePeriod=30
Aug 04 21:20:03.504072 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.504012    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:20:03.547964 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.547931    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66dae5ff-f41d-496a-bd91-d11fcee34b83-maas-api-tls\") pod \"66dae5ff-f41d-496a-bd91-d11fcee34b83\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") "
Aug 04 21:20:03.548150 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.547997    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66dae5ff-f41d-496a-bd91-d11fcee34b83-openshift-service-ca\") pod \"66dae5ff-f41d-496a-bd91-d11fcee34b83\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") "
Aug 04 21:20:03.548150 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.548023    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmtbd\" (UniqueName: \"kubernetes.io/projected/66dae5ff-f41d-496a-bd91-d11fcee34b83-kube-api-access-kmtbd\") pod \"66dae5ff-f41d-496a-bd91-d11fcee34b83\" (UID: \"66dae5ff-f41d-496a-bd91-d11fcee34b83\") "
Aug 04 21:20:03.548471 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.548440    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66dae5ff-f41d-496a-bd91-d11fcee34b83-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "66dae5ff-f41d-496a-bd91-d11fcee34b83" (UID: "66dae5ff-f41d-496a-bd91-d11fcee34b83"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:20:03.550277 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.550253    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66dae5ff-f41d-496a-bd91-d11fcee34b83-kube-api-access-kmtbd" (OuterVolumeSpecName: "kube-api-access-kmtbd") pod "66dae5ff-f41d-496a-bd91-d11fcee34b83" (UID: "66dae5ff-f41d-496a-bd91-d11fcee34b83"). InnerVolumeSpecName "kube-api-access-kmtbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:20:03.550277 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.550255    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66dae5ff-f41d-496a-bd91-d11fcee34b83-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "66dae5ff-f41d-496a-bd91-d11fcee34b83" (UID: "66dae5ff-f41d-496a-bd91-d11fcee34b83"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:20:03.648710 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.648676    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66dae5ff-f41d-496a-bd91-d11fcee34b83-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:20:03.648710 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.648703    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmtbd\" (UniqueName: \"kubernetes.io/projected/66dae5ff-f41d-496a-bd91-d11fcee34b83-kube-api-access-kmtbd\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:20:03.648903 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:03.648718    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66dae5ff-f41d-496a-bd91-d11fcee34b83-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:20:04.287463 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:04.287426    2563 generic.go:356] "Generic (PLEG): container finished" podID="66dae5ff-f41d-496a-bd91-d11fcee34b83" containerID="a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f" exitCode=0
Aug 04 21:20:04.287893 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:04.287474    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"
Aug 04 21:20:04.287893 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:04.287517    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh" event={"ID":"66dae5ff-f41d-496a-bd91-d11fcee34b83","Type":"ContainerDied","Data":"a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f"}
Aug 04 21:20:04.287893 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:04.287571    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh" event={"ID":"66dae5ff-f41d-496a-bd91-d11fcee34b83","Type":"ContainerDied","Data":"b4797c0a3fc3829a9a7fb8d0ff19a2381a4d5c38c50865eea379c981153d9d79"}
Aug 04 21:20:04.287893 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:04.287597    2563 scope.go:117] "RemoveContainer" containerID="a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f"
Aug 04 21:20:04.297533 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:04.297516    2563 scope.go:117] "RemoveContainer" containerID="a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f"
Aug 04 21:20:04.297814 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:20:04.297791    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f\": container with ID starting with a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f not found: ID does not exist" containerID="a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f"
Aug 04 21:20:04.297889 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:04.297825    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f"} err="failed to get container status \"a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f\": rpc error: code = NotFound desc = could not find container \"a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f\": container with ID starting with a5d7bf6ca8ca5992f1241ee38817bbc36d11a2f0e0bfc7b7349dfd17ae95a47f not found: ID does not exist"
Aug 04 21:20:04.320151 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:04.320126    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"]
Aug 04 21:20:04.329788 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:04.329769    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d1f5ec52-674d5ffb74-x7kjh"]
Aug 04 21:20:05.243066 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:20:05.243034    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66dae5ff-f41d-496a-bd91-d11fcee34b83" path="/var/lib/kubelet/pods/66dae5ff-f41d-496a-bd91-d11fcee34b83/volumes"
Aug 04 21:22:04.417575 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.417534    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"]
Aug 04 21:22:04.418144 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.418123    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66dae5ff-f41d-496a-bd91-d11fcee34b83" containerName="maas-api"
Aug 04 21:22:04.418192 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.418149    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="66dae5ff-f41d-496a-bd91-d11fcee34b83" containerName="maas-api"
Aug 04 21:22:04.418271 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.418259    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="66dae5ff-f41d-496a-bd91-d11fcee34b83" containerName="maas-api"
Aug 04 21:22:04.421208 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.421186    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:04.423836 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.423819    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-19ad13-serving-cert\""
Aug 04 21:22:04.432932 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.432908    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"]
Aug 04 21:22:04.579160 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.579125    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e3fe400-c081-4b14-88ee-569aa47a2947-openshift-service-ca\") pod \"maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:04.579365 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.579181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e3fe400-c081-4b14-88ee-569aa47a2947-maas-api-tls\") pod \"maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:04.579365 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.579305    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rchfn\" (UniqueName: \"kubernetes.io/projected/1e3fe400-c081-4b14-88ee-569aa47a2947-kube-api-access-rchfn\") pod \"maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:04.680621 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.680539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e3fe400-c081-4b14-88ee-569aa47a2947-openshift-service-ca\") pod \"maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:04.680621 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.680584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e3fe400-c081-4b14-88ee-569aa47a2947-maas-api-tls\") pod \"maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:04.680800 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.680634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rchfn\" (UniqueName: \"kubernetes.io/projected/1e3fe400-c081-4b14-88ee-569aa47a2947-kube-api-access-rchfn\") pod \"maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:04.681196 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.681165    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e3fe400-c081-4b14-88ee-569aa47a2947-openshift-service-ca\") pod \"maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:04.683096 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.683076    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e3fe400-c081-4b14-88ee-569aa47a2947-maas-api-tls\") pod \"maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:04.689161 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.689137    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rchfn\" (UniqueName: \"kubernetes.io/projected/1e3fe400-c081-4b14-88ee-569aa47a2947-kube-api-access-rchfn\") pod \"maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:04.733436 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.733411    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:04.874020 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.873983    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"]
Aug 04 21:22:04.875630 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:22:04.875603    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3fe400_c081_4b14_88ee_569aa47a2947.slice/crio-37f344aef6d25ebfd4c5618698895a24eab8021e060675cae8373872d83c3ccd WatchSource:0}: Error finding container 37f344aef6d25ebfd4c5618698895a24eab8021e060675cae8373872d83c3ccd: Status 404 returned error can't find the container with id 37f344aef6d25ebfd4c5618698895a24eab8021e060675cae8373872d83c3ccd
Aug 04 21:22:04.935960 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:04.935687    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq" event={"ID":"1e3fe400-c081-4b14-88ee-569aa47a2947","Type":"ContainerStarted","Data":"37f344aef6d25ebfd4c5618698895a24eab8021e060675cae8373872d83c3ccd"}
Aug 04 21:22:05.942601 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:05.942564    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq" event={"ID":"1e3fe400-c081-4b14-88ee-569aa47a2947","Type":"ContainerStarted","Data":"2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b"}
Aug 04 21:22:05.943039 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:05.942757    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:05.962135 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:05.962077    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq" podStartSLOduration=1.4848421520000001 podStartE2EDuration="1.962060294s" totalImagesPullingTime="477.218142ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:22:05.958700677 +0000 UTC m=+2473.202319983" watchObservedRunningTime="2026-08-04 21:22:05.962060294 +0000 UTC m=+2473.205679614"
Aug 04 21:22:11.958135 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:11.958103    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:22:32.530052 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.530017    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"]
Aug 04 21:22:32.533625 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.533606    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:32.536172 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.536153    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-4c3747-serving-cert\""
Aug 04 21:22:32.545972 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.545952    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"]
Aug 04 21:22:32.621577 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.621542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caa8eb43-d3b5-48ce-8929-2e801d193721-maas-api-tls\") pod \"maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:32.621577 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.621575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltwn8\" (UniqueName: \"kubernetes.io/projected/caa8eb43-d3b5-48ce-8929-2e801d193721-kube-api-access-ltwn8\") pod \"maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:32.621810 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.621652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa8eb43-d3b5-48ce-8929-2e801d193721-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:32.722649 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.722610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caa8eb43-d3b5-48ce-8929-2e801d193721-maas-api-tls\") pod \"maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:32.722649 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.722654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltwn8\" (UniqueName: \"kubernetes.io/projected/caa8eb43-d3b5-48ce-8929-2e801d193721-kube-api-access-ltwn8\") pod \"maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:32.722918 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.722753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa8eb43-d3b5-48ce-8929-2e801d193721-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:32.723464 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.723441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa8eb43-d3b5-48ce-8929-2e801d193721-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:32.725346 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.725308    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caa8eb43-d3b5-48ce-8929-2e801d193721-maas-api-tls\") pod \"maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:32.730837 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.730814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltwn8\" (UniqueName: \"kubernetes.io/projected/caa8eb43-d3b5-48ce-8929-2e801d193721-kube-api-access-ltwn8\") pod \"maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:32.844474 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.844388    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:32.990924 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:32.990876    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"]
Aug 04 21:22:33.086953 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:33.086921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt" event={"ID":"caa8eb43-d3b5-48ce-8929-2e801d193721","Type":"ContainerStarted","Data":"05432916451d19376dd13e974232479efeafc10d3963f3481ed8b556de4dd19f"}
Aug 04 21:22:34.094386 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:34.094351    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt" event={"ID":"caa8eb43-d3b5-48ce-8929-2e801d193721","Type":"ContainerStarted","Data":"921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9"}
Aug 04 21:22:34.094786 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:34.094534    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:22:34.112465 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:34.112420    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt" podStartSLOduration=1.833265368 podStartE2EDuration="2.112404438s" totalImagesPullingTime="279.13907ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:22:34.111299586 +0000 UTC m=+2501.354918872" watchObservedRunningTime="2026-08-04 21:22:34.112404438 +0000 UTC m=+2501.356023719"
Aug 04 21:22:40.109015 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:22:40.108982    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:24:21.588125 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:21.588037    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"]
Aug 04 21:24:21.588691 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:21.588287    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq" podUID="1e3fe400-c081-4b14-88ee-569aa47a2947" containerName="maas-api" containerID="cri-o://2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b" gracePeriod=30
Aug 04 21:24:21.858520 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:21.858497    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:24:21.979580 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:21.979543    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rchfn\" (UniqueName: \"kubernetes.io/projected/1e3fe400-c081-4b14-88ee-569aa47a2947-kube-api-access-rchfn\") pod \"1e3fe400-c081-4b14-88ee-569aa47a2947\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") "
Aug 04 21:24:21.979758 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:21.979613    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e3fe400-c081-4b14-88ee-569aa47a2947-maas-api-tls\") pod \"1e3fe400-c081-4b14-88ee-569aa47a2947\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") "
Aug 04 21:24:21.979758 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:21.979651    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e3fe400-c081-4b14-88ee-569aa47a2947-openshift-service-ca\") pod \"1e3fe400-c081-4b14-88ee-569aa47a2947\" (UID: \"1e3fe400-c081-4b14-88ee-569aa47a2947\") "
Aug 04 21:24:21.980026 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:21.979999    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e3fe400-c081-4b14-88ee-569aa47a2947-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1e3fe400-c081-4b14-88ee-569aa47a2947" (UID: "1e3fe400-c081-4b14-88ee-569aa47a2947"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:24:21.981702 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:21.981681    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e3fe400-c081-4b14-88ee-569aa47a2947-kube-api-access-rchfn" (OuterVolumeSpecName: "kube-api-access-rchfn") pod "1e3fe400-c081-4b14-88ee-569aa47a2947" (UID: "1e3fe400-c081-4b14-88ee-569aa47a2947"). InnerVolumeSpecName "kube-api-access-rchfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:24:21.981785 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:21.981763    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e3fe400-c081-4b14-88ee-569aa47a2947-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1e3fe400-c081-4b14-88ee-569aa47a2947" (UID: "1e3fe400-c081-4b14-88ee-569aa47a2947"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:24:22.080747 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.080713    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e3fe400-c081-4b14-88ee-569aa47a2947-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:24:22.080747 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.080743    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e3fe400-c081-4b14-88ee-569aa47a2947-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:24:22.080747 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.080754    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rchfn\" (UniqueName: \"kubernetes.io/projected/1e3fe400-c081-4b14-88ee-569aa47a2947-kube-api-access-rchfn\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:24:22.681100 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.681059    2563 generic.go:356] "Generic (PLEG): container finished" podID="1e3fe400-c081-4b14-88ee-569aa47a2947" containerID="2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b" exitCode=0
Aug 04 21:24:22.681608 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.681127    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"
Aug 04 21:24:22.681608 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.681141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq" event={"ID":"1e3fe400-c081-4b14-88ee-569aa47a2947","Type":"ContainerDied","Data":"2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b"}
Aug 04 21:24:22.681608 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.681204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq" event={"ID":"1e3fe400-c081-4b14-88ee-569aa47a2947","Type":"ContainerDied","Data":"37f344aef6d25ebfd4c5618698895a24eab8021e060675cae8373872d83c3ccd"}
Aug 04 21:24:22.681608 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.681228    2563 scope.go:117] "RemoveContainer" containerID="2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b"
Aug 04 21:24:22.693270 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.692815    2563 scope.go:117] "RemoveContainer" containerID="2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b"
Aug 04 21:24:22.693753 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:24:22.693723    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b\": container with ID starting with 2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b not found: ID does not exist" containerID="2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b"
Aug 04 21:24:22.693888 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.693762    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b"} err="failed to get container status \"2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b\": rpc error: code = NotFound desc = could not find container \"2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b\": container with ID starting with 2d44148133c7953b92dcfd915daef4458516f6bc8e93a6556e5cdfb2788cb28b not found: ID does not exist"
Aug 04 21:24:22.706279 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.706245    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"]
Aug 04 21:24:22.709848 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:22.709825    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19ad13-f767dbd9d-mtblq"]
Aug 04 21:24:23.244404 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:23.244374    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e3fe400-c081-4b14-88ee-569aa47a2947" path="/var/lib/kubelet/pods/1e3fe400-c081-4b14-88ee-569aa47a2947/volumes"
Aug 04 21:24:44.491623 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:44.491592    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"]
Aug 04 21:24:44.801862 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:44.801762    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt" podUID="caa8eb43-d3b5-48ce-8929-2e801d193721" containerName="maas-api" containerID="cri-o://921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9" gracePeriod=30
Aug 04 21:24:45.047475 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.047452    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:24:45.186676 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.186643    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caa8eb43-d3b5-48ce-8929-2e801d193721-maas-api-tls\") pod \"caa8eb43-d3b5-48ce-8929-2e801d193721\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") "
Aug 04 21:24:45.186676 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.186676    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltwn8\" (UniqueName: \"kubernetes.io/projected/caa8eb43-d3b5-48ce-8929-2e801d193721-kube-api-access-ltwn8\") pod \"caa8eb43-d3b5-48ce-8929-2e801d193721\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") "
Aug 04 21:24:45.186876 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.186700    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa8eb43-d3b5-48ce-8929-2e801d193721-openshift-service-ca\") pod \"caa8eb43-d3b5-48ce-8929-2e801d193721\" (UID: \"caa8eb43-d3b5-48ce-8929-2e801d193721\") "
Aug 04 21:24:45.187102 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.187082    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caa8eb43-d3b5-48ce-8929-2e801d193721-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "caa8eb43-d3b5-48ce-8929-2e801d193721" (UID: "caa8eb43-d3b5-48ce-8929-2e801d193721"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:24:45.188864 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.188842    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caa8eb43-d3b5-48ce-8929-2e801d193721-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "caa8eb43-d3b5-48ce-8929-2e801d193721" (UID: "caa8eb43-d3b5-48ce-8929-2e801d193721"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:24:45.188960 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.188867    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caa8eb43-d3b5-48ce-8929-2e801d193721-kube-api-access-ltwn8" (OuterVolumeSpecName: "kube-api-access-ltwn8") pod "caa8eb43-d3b5-48ce-8929-2e801d193721" (UID: "caa8eb43-d3b5-48ce-8929-2e801d193721"). InnerVolumeSpecName "kube-api-access-ltwn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:24:45.288040 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.288010    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caa8eb43-d3b5-48ce-8929-2e801d193721-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:24:45.288040 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.288036    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltwn8\" (UniqueName: \"kubernetes.io/projected/caa8eb43-d3b5-48ce-8929-2e801d193721-kube-api-access-ltwn8\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:24:45.288040 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.288046    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa8eb43-d3b5-48ce-8929-2e801d193721-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:24:45.808187 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.808155    2563 generic.go:356] "Generic (PLEG): container finished" podID="caa8eb43-d3b5-48ce-8929-2e801d193721" containerID="921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9" exitCode=0
Aug 04 21:24:45.808744 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.808213    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"
Aug 04 21:24:45.808744 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.808258    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt" event={"ID":"caa8eb43-d3b5-48ce-8929-2e801d193721","Type":"ContainerDied","Data":"921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9"}
Aug 04 21:24:45.808744 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.808343    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt" event={"ID":"caa8eb43-d3b5-48ce-8929-2e801d193721","Type":"ContainerDied","Data":"05432916451d19376dd13e974232479efeafc10d3963f3481ed8b556de4dd19f"}
Aug 04 21:24:45.808744 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.808378    2563 scope.go:117] "RemoveContainer" containerID="921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9"
Aug 04 21:24:45.817668 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.817634    2563 scope.go:117] "RemoveContainer" containerID="921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9"
Aug 04 21:24:45.817919 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:24:45.817897    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9\": container with ID starting with 921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9 not found: ID does not exist" containerID="921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9"
Aug 04 21:24:45.817974 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.817929    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9"} err="failed to get container status \"921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9\": rpc error: code = NotFound desc = could not find container \"921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9\": container with ID starting with 921b3181d378e7baae9d689af89977d59accd0dc2b7b3c3ac73eecc89e0046e9 not found: ID does not exist"
Aug 04 21:24:45.826002 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.825982    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"]
Aug 04 21:24:45.831617 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:45.831597    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c3747-5f8bd8798d-mngrt"]
Aug 04 21:24:47.243946 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:47.243903    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caa8eb43-d3b5-48ce-8929-2e801d193721" path="/var/lib/kubelet/pods/caa8eb43-d3b5-48ce-8929-2e801d193721/volumes"
Aug 04 21:24:56.247710 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.247672    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"]
Aug 04 21:24:56.248459 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.248432    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caa8eb43-d3b5-48ce-8929-2e801d193721" containerName="maas-api"
Aug 04 21:24:56.248459 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.248458    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="caa8eb43-d3b5-48ce-8929-2e801d193721" containerName="maas-api"
Aug 04 21:24:56.248655 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.248482    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e3fe400-c081-4b14-88ee-569aa47a2947" containerName="maas-api"
Aug 04 21:24:56.248655 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.248492    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e3fe400-c081-4b14-88ee-569aa47a2947" containerName="maas-api"
Aug 04 21:24:56.248655 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.248604    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e3fe400-c081-4b14-88ee-569aa47a2947" containerName="maas-api"
Aug 04 21:24:56.248655 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.248616    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="caa8eb43-d3b5-48ce-8929-2e801d193721" containerName="maas-api"
Aug 04 21:24:56.253923 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.253901    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:56.256811 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.256790    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a8febf-serving-cert\""
Aug 04 21:24:56.268410 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.268379    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"]
Aug 04 21:24:56.282003 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.281967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f916e38-e321-4592-bcf0-a9905491903a-maas-api-tls\") pod \"maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:56.282213 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.282134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7w82\" (UniqueName: \"kubernetes.io/projected/2f916e38-e321-4592-bcf0-a9905491903a-kube-api-access-t7w82\") pod \"maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:56.282213 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.282166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f916e38-e321-4592-bcf0-a9905491903a-openshift-service-ca\") pod \"maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:56.383246 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.383208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f916e38-e321-4592-bcf0-a9905491903a-maas-api-tls\") pod \"maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:56.383246 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.383249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7w82\" (UniqueName: \"kubernetes.io/projected/2f916e38-e321-4592-bcf0-a9905491903a-kube-api-access-t7w82\") pod \"maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:56.383499 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.383381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f916e38-e321-4592-bcf0-a9905491903a-openshift-service-ca\") pod \"maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:56.383499 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:24:56.383391    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-serving-cert: secret "maas-api-e2e-api-a-a8febf-serving-cert" not found
Aug 04 21:24:56.383602 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:24:56.383589    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f916e38-e321-4592-bcf0-a9905491903a-maas-api-tls podName:2f916e38-e321-4592-bcf0-a9905491903a nodeName:}" failed. No retries permitted until 2026-08-04 21:24:56.88356799 +0000 UTC m=+2644.127187261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f916e38-e321-4592-bcf0-a9905491903a-maas-api-tls") pod "maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4" (UID: "2f916e38-e321-4592-bcf0-a9905491903a") : secret "maas-api-e2e-api-a-a8febf-serving-cert" not found
Aug 04 21:24:56.383977 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.383956    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f916e38-e321-4592-bcf0-a9905491903a-openshift-service-ca\") pod \"maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:56.394798 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.394763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7w82\" (UniqueName: \"kubernetes.io/projected/2f916e38-e321-4592-bcf0-a9905491903a-kube-api-access-t7w82\") pod \"maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:56.888611 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.888576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f916e38-e321-4592-bcf0-a9905491903a-maas-api-tls\") pod \"maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:56.891731 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:56.891698    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f916e38-e321-4592-bcf0-a9905491903a-maas-api-tls\") pod \"maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:57.163652 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:57.163556    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:57.325251 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:57.325226    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"]
Aug 04 21:24:57.327931 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:24:57.327904    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f916e38_e321_4592_bcf0_a9905491903a.slice/crio-214581c3cb746bc9df2d1d9c5b09d22c85af19e1931acabed052f30d9c61c9b9 WatchSource:0}: Error finding container 214581c3cb746bc9df2d1d9c5b09d22c85af19e1931acabed052f30d9c61c9b9: Status 404 returned error can't find the container with id 214581c3cb746bc9df2d1d9c5b09d22c85af19e1931acabed052f30d9c61c9b9
Aug 04 21:24:57.329277 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:57.329259    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 21:24:57.885862 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:57.885825    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4" event={"ID":"2f916e38-e321-4592-bcf0-a9905491903a","Type":"ContainerStarted","Data":"214581c3cb746bc9df2d1d9c5b09d22c85af19e1931acabed052f30d9c61c9b9"}
Aug 04 21:24:58.893018 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:58.892983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4" event={"ID":"2f916e38-e321-4592-bcf0-a9905491903a","Type":"ContainerStarted","Data":"6257c3265ea4261f4270f93feb3a59b1f66faf9890c82fb2b958edb4cb2d8b94"}
Aug 04 21:24:58.893472 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:58.893117    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:24:58.910453 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:24:58.910406    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4" podStartSLOduration=2.348457335 podStartE2EDuration="2.910393128s" totalImagesPullingTime="561.935793ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:24:58.909162416 +0000 UTC m=+2646.152781699" watchObservedRunningTime="2026-08-04 21:24:58.910393128 +0000 UTC m=+2646.154012409"
Aug 04 21:25:04.906684 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:04.906647    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:25:25.482357 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.482303    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"]
Aug 04 21:25:25.486286 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.486263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:25.488834 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.488815    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-c4ea15-serving-cert\""
Aug 04 21:25:25.498076 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.498052    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"]
Aug 04 21:25:25.641393 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.641353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37b10052-0ad0-4ddb-aeed-da2d584d474d-maas-api-tls\") pod \"maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:25.641586 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.641402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b10052-0ad0-4ddb-aeed-da2d584d474d-openshift-service-ca\") pod \"maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:25.641586 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.641495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2x8h\" (UniqueName: \"kubernetes.io/projected/37b10052-0ad0-4ddb-aeed-da2d584d474d-kube-api-access-d2x8h\") pod \"maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:25.742263 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.742175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2x8h\" (UniqueName: \"kubernetes.io/projected/37b10052-0ad0-4ddb-aeed-da2d584d474d-kube-api-access-d2x8h\") pod \"maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:25.742263 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.742249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37b10052-0ad0-4ddb-aeed-da2d584d474d-maas-api-tls\") pod \"maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:25.742515 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.742270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b10052-0ad0-4ddb-aeed-da2d584d474d-openshift-service-ca\") pod \"maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:25.742857 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.742836    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b10052-0ad0-4ddb-aeed-da2d584d474d-openshift-service-ca\") pod \"maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:25.744932 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.744907    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37b10052-0ad0-4ddb-aeed-da2d584d474d-maas-api-tls\") pod \"maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:25.751175 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.751149    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2x8h\" (UniqueName: \"kubernetes.io/projected/37b10052-0ad0-4ddb-aeed-da2d584d474d-kube-api-access-d2x8h\") pod \"maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:25.798013 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.797988    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:25.935540 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:25.935509    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"]
Aug 04 21:25:25.936823 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:25:25.936797    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b10052_0ad0_4ddb_aeed_da2d584d474d.slice/crio-c59800ca5049479aee50d5a4b75b21b5affb725a4a27aeac01c78314bc3da5b8 WatchSource:0}: Error finding container c59800ca5049479aee50d5a4b75b21b5affb725a4a27aeac01c78314bc3da5b8: Status 404 returned error can't find the container with id c59800ca5049479aee50d5a4b75b21b5affb725a4a27aeac01c78314bc3da5b8
Aug 04 21:25:26.041148 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:26.041047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd" event={"ID":"37b10052-0ad0-4ddb-aeed-da2d584d474d","Type":"ContainerStarted","Data":"c59800ca5049479aee50d5a4b75b21b5affb725a4a27aeac01c78314bc3da5b8"}
Aug 04 21:25:27.047914 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:27.047878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd" event={"ID":"37b10052-0ad0-4ddb-aeed-da2d584d474d","Type":"ContainerStarted","Data":"7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59"}
Aug 04 21:25:27.048403 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:27.048008    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:25:27.069809 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:27.069758    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd" podStartSLOduration=1.680608441 podStartE2EDuration="2.06974122s" totalImagesPullingTime="389.132779ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:25:27.06924425 +0000 UTC m=+2674.312863557" watchObservedRunningTime="2026-08-04 21:25:27.06974122 +0000 UTC m=+2674.313360502"
Aug 04 21:25:33.059143 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:25:33.059115    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:26:02.025070 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.024975    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"]
Aug 04 21:26:02.025607 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.025269    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4" podUID="2f916e38-e321-4592-bcf0-a9905491903a" containerName="maas-api" containerID="cri-o://6257c3265ea4261f4270f93feb3a59b1f66faf9890c82fb2b958edb4cb2d8b94" gracePeriod=30
Aug 04 21:26:02.237612 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.237583    2563 generic.go:356] "Generic (PLEG): container finished" podID="2f916e38-e321-4592-bcf0-a9905491903a" containerID="6257c3265ea4261f4270f93feb3a59b1f66faf9890c82fb2b958edb4cb2d8b94" exitCode=0
Aug 04 21:26:02.237612 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.237603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4" event={"ID":"2f916e38-e321-4592-bcf0-a9905491903a","Type":"ContainerDied","Data":"6257c3265ea4261f4270f93feb3a59b1f66faf9890c82fb2b958edb4cb2d8b94"}
Aug 04 21:26:02.282273 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.282208    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:26:02.369037 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.369006    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f916e38-e321-4592-bcf0-a9905491903a-maas-api-tls\") pod \"2f916e38-e321-4592-bcf0-a9905491903a\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") "
Aug 04 21:26:02.369223 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.369053    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7w82\" (UniqueName: \"kubernetes.io/projected/2f916e38-e321-4592-bcf0-a9905491903a-kube-api-access-t7w82\") pod \"2f916e38-e321-4592-bcf0-a9905491903a\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") "
Aug 04 21:26:02.369223 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.369152    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f916e38-e321-4592-bcf0-a9905491903a-openshift-service-ca\") pod \"2f916e38-e321-4592-bcf0-a9905491903a\" (UID: \"2f916e38-e321-4592-bcf0-a9905491903a\") "
Aug 04 21:26:02.369576 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.369546    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f916e38-e321-4592-bcf0-a9905491903a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f916e38-e321-4592-bcf0-a9905491903a" (UID: "2f916e38-e321-4592-bcf0-a9905491903a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:26:02.371161 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.371139    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f916e38-e321-4592-bcf0-a9905491903a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f916e38-e321-4592-bcf0-a9905491903a" (UID: "2f916e38-e321-4592-bcf0-a9905491903a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:26:02.371303 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.371281    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f916e38-e321-4592-bcf0-a9905491903a-kube-api-access-t7w82" (OuterVolumeSpecName: "kube-api-access-t7w82") pod "2f916e38-e321-4592-bcf0-a9905491903a" (UID: "2f916e38-e321-4592-bcf0-a9905491903a"). InnerVolumeSpecName "kube-api-access-t7w82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:26:02.470663 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.470620    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f916e38-e321-4592-bcf0-a9905491903a-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:26:02.470663 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.470657    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f916e38-e321-4592-bcf0-a9905491903a-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:26:02.470663 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:02.470666    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7w82\" (UniqueName: \"kubernetes.io/projected/2f916e38-e321-4592-bcf0-a9905491903a-kube-api-access-t7w82\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:26:03.243528 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:03.243492    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4" event={"ID":"2f916e38-e321-4592-bcf0-a9905491903a","Type":"ContainerDied","Data":"214581c3cb746bc9df2d1d9c5b09d22c85af19e1931acabed052f30d9c61c9b9"}
Aug 04 21:26:03.243939 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:03.243546    2563 scope.go:117] "RemoveContainer" containerID="6257c3265ea4261f4270f93feb3a59b1f66faf9890c82fb2b958edb4cb2d8b94"
Aug 04 21:26:03.243939 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:03.243506    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"
Aug 04 21:26:03.266508 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:03.266482    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"]
Aug 04 21:26:03.269033 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:03.269013    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8febf-869dfb9cc6-xb2p4"]
Aug 04 21:26:05.243526 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:05.243484    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f916e38-e321-4592-bcf0-a9905491903a" path="/var/lib/kubelet/pods/2f916e38-e321-4592-bcf0-a9905491903a/volumes"
Aug 04 21:26:24.594167 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.594123    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"]
Aug 04 21:26:24.594703 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.594431    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd" podUID="37b10052-0ad0-4ddb-aeed-da2d584d474d" containerName="maas-api" containerID="cri-o://7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59" gracePeriod=30
Aug 04 21:26:24.854490 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.854433    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:26:24.874680 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.874613    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b10052-0ad0-4ddb-aeed-da2d584d474d-openshift-service-ca\") pod \"37b10052-0ad0-4ddb-aeed-da2d584d474d\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") "
Aug 04 21:26:24.874820 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.874751    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37b10052-0ad0-4ddb-aeed-da2d584d474d-maas-api-tls\") pod \"37b10052-0ad0-4ddb-aeed-da2d584d474d\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") "
Aug 04 21:26:24.874820 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.874782    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2x8h\" (UniqueName: \"kubernetes.io/projected/37b10052-0ad0-4ddb-aeed-da2d584d474d-kube-api-access-d2x8h\") pod \"37b10052-0ad0-4ddb-aeed-da2d584d474d\" (UID: \"37b10052-0ad0-4ddb-aeed-da2d584d474d\") "
Aug 04 21:26:24.876289 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.875601    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37b10052-0ad0-4ddb-aeed-da2d584d474d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "37b10052-0ad0-4ddb-aeed-da2d584d474d" (UID: "37b10052-0ad0-4ddb-aeed-da2d584d474d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:26:24.877660 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.877634    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37b10052-0ad0-4ddb-aeed-da2d584d474d-kube-api-access-d2x8h" (OuterVolumeSpecName: "kube-api-access-d2x8h") pod "37b10052-0ad0-4ddb-aeed-da2d584d474d" (UID: "37b10052-0ad0-4ddb-aeed-da2d584d474d"). InnerVolumeSpecName "kube-api-access-d2x8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:26:24.877914 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.877891    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37b10052-0ad0-4ddb-aeed-da2d584d474d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "37b10052-0ad0-4ddb-aeed-da2d584d474d" (UID: "37b10052-0ad0-4ddb-aeed-da2d584d474d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:26:24.975595 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.975565    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b10052-0ad0-4ddb-aeed-da2d584d474d-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:26:24.975595 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.975590    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37b10052-0ad0-4ddb-aeed-da2d584d474d-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:26:24.975595 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:24.975599    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2x8h\" (UniqueName: \"kubernetes.io/projected/37b10052-0ad0-4ddb-aeed-da2d584d474d-kube-api-access-d2x8h\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:26:25.365773 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:25.365733    2563 generic.go:356] "Generic (PLEG): container finished" podID="37b10052-0ad0-4ddb-aeed-da2d584d474d" containerID="7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59" exitCode=0
Aug 04 21:26:25.365938 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:25.365793    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"
Aug 04 21:26:25.365938 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:25.365814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd" event={"ID":"37b10052-0ad0-4ddb-aeed-da2d584d474d","Type":"ContainerDied","Data":"7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59"}
Aug 04 21:26:25.365938 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:25.365897    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd" event={"ID":"37b10052-0ad0-4ddb-aeed-da2d584d474d","Type":"ContainerDied","Data":"c59800ca5049479aee50d5a4b75b21b5affb725a4a27aeac01c78314bc3da5b8"}
Aug 04 21:26:25.365938 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:25.365926    2563 scope.go:117] "RemoveContainer" containerID="7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59"
Aug 04 21:26:25.375160 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:25.375142    2563 scope.go:117] "RemoveContainer" containerID="7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59"
Aug 04 21:26:25.375436 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:26:25.375417    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59\": container with ID starting with 7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59 not found: ID does not exist" containerID="7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59"
Aug 04 21:26:25.375512 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:25.375444    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59"} err="failed to get container status \"7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59\": rpc error: code = NotFound desc = could not find container \"7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59\": container with ID starting with 7d441a2505b17ccc34571ccd1e51a64c9abab3d3ae79afdd1ff025b87d85cc59 not found: ID does not exist"
Aug 04 21:26:25.383070 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:25.383051    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"]
Aug 04 21:26:25.388758 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:25.388735    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c4ea15-bb965c65b-cgfwd"]
Aug 04 21:26:27.243694 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:27.243658    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37b10052-0ad0-4ddb-aeed-da2d584d474d" path="/var/lib/kubelet/pods/37b10052-0ad0-4ddb-aeed-da2d584d474d/volumes"
Aug 04 21:26:36.711463 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.711428    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"]
Aug 04 21:26:36.713863 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.711810    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f916e38-e321-4592-bcf0-a9905491903a" containerName="maas-api"
Aug 04 21:26:36.713863 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.711822    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f916e38-e321-4592-bcf0-a9905491903a" containerName="maas-api"
Aug 04 21:26:36.713863 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.711841    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b10052-0ad0-4ddb-aeed-da2d584d474d" containerName="maas-api"
Aug 04 21:26:36.713863 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.711846    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b10052-0ad0-4ddb-aeed-da2d584d474d" containerName="maas-api"
Aug 04 21:26:36.713863 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.711904    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f916e38-e321-4592-bcf0-a9905491903a" containerName="maas-api"
Aug 04 21:26:36.713863 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.711914    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="37b10052-0ad0-4ddb-aeed-da2d584d474d" containerName="maas-api"
Aug 04 21:26:36.714838 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.714824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:36.717172 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.717150    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-a189ec-serving-cert\""
Aug 04 21:26:36.725487 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.725461    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"]
Aug 04 21:26:36.776373 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.776296    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24b760e1-3098-4eae-9680-de234ca82947-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:36.776562 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.776387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq7cw\" (UniqueName: \"kubernetes.io/projected/24b760e1-3098-4eae-9680-de234ca82947-kube-api-access-mq7cw\") pod \"maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:36.776562 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.776468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls\") pod \"maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:36.877065 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.877030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24b760e1-3098-4eae-9680-de234ca82947-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:36.877251 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.877075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq7cw\" (UniqueName: \"kubernetes.io/projected/24b760e1-3098-4eae-9680-de234ca82947-kube-api-access-mq7cw\") pod \"maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:36.877251 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.877127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls\") pod \"maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:36.877432 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:26:36.877282    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-serving-cert: secret "maas-api-e2e-shared-a-a189ec-serving-cert" not found
Aug 04 21:26:36.877432 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:26:36.877380    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls podName:24b760e1-3098-4eae-9680-de234ca82947 nodeName:}" failed. No retries permitted until 2026-08-04 21:26:37.377357305 +0000 UTC m=+2744.620976570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls") pod "maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79" (UID: "24b760e1-3098-4eae-9680-de234ca82947") : secret "maas-api-e2e-shared-a-a189ec-serving-cert" not found
Aug 04 21:26:36.877780 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.877758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24b760e1-3098-4eae-9680-de234ca82947-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:36.886129 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:36.886099    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq7cw\" (UniqueName: \"kubernetes.io/projected/24b760e1-3098-4eae-9680-de234ca82947-kube-api-access-mq7cw\") pod \"maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:37.382202 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:37.382170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls\") pod \"maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:37.382389 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:26:37.382335    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-serving-cert: secret "maas-api-e2e-shared-a-a189ec-serving-cert" not found
Aug 04 21:26:37.382440 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:26:37.382411    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls podName:24b760e1-3098-4eae-9680-de234ca82947 nodeName:}" failed. No retries permitted until 2026-08-04 21:26:38.382393656 +0000 UTC m=+2745.626012919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls") pod "maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79" (UID: "24b760e1-3098-4eae-9680-de234ca82947") : secret "maas-api-e2e-shared-a-a189ec-serving-cert" not found
Aug 04 21:26:38.391184 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:38.391145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls\") pod \"maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:38.393915 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:38.393886    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls\") pod \"maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:38.526303 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:38.526260    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:38.877346 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:38.877299    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"]
Aug 04 21:26:38.880726 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:26:38.880691    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b760e1_3098_4eae_9680_de234ca82947.slice/crio-a9d0db01207cc7f96ef2339b641863c64ec21153f9b9b2cf108ffa97a510104a WatchSource:0}: Error finding container a9d0db01207cc7f96ef2339b641863c64ec21153f9b9b2cf108ffa97a510104a: Status 404 returned error can't find the container with id a9d0db01207cc7f96ef2339b641863c64ec21153f9b9b2cf108ffa97a510104a
Aug 04 21:26:39.450060 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:39.450026    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79" event={"ID":"24b760e1-3098-4eae-9680-de234ca82947","Type":"ContainerStarted","Data":"25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a"}
Aug 04 21:26:39.450530 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:39.450078    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79" event={"ID":"24b760e1-3098-4eae-9680-de234ca82947","Type":"ContainerStarted","Data":"a9d0db01207cc7f96ef2339b641863c64ec21153f9b9b2cf108ffa97a510104a"}
Aug 04 21:26:39.450530 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:39.450130    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:26:39.466982 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:39.466940    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79" podStartSLOduration=3.109496761 podStartE2EDuration="3.466924516s" totalImagesPullingTime="357.427755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:26:39.46595946 +0000 UTC m=+2746.709578766" watchObservedRunningTime="2026-08-04 21:26:39.466924516 +0000 UTC m=+2746.710543818"
Aug 04 21:26:45.460754 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:26:45.460725    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:27:04.948690 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:04.948657    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"]
Aug 04 21:27:04.952034 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:04.952014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:04.954418 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:04.954395    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-cd644a-serving-cert\""
Aug 04 21:27:04.963547 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:04.963527    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"]
Aug 04 21:27:05.127995 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.127960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57d3d212-b9ce-40da-ac77-367f135fce62-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:05.127995 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.128003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57d3d212-b9ce-40da-ac77-367f135fce62-maas-api-tls\") pod \"maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:05.128210 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.128025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcl7q\" (UniqueName: \"kubernetes.io/projected/57d3d212-b9ce-40da-ac77-367f135fce62-kube-api-access-gcl7q\") pod \"maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:05.229093 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.229011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57d3d212-b9ce-40da-ac77-367f135fce62-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:05.229093 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.229048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57d3d212-b9ce-40da-ac77-367f135fce62-maas-api-tls\") pod \"maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:05.229093 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.229067    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcl7q\" (UniqueName: \"kubernetes.io/projected/57d3d212-b9ce-40da-ac77-367f135fce62-kube-api-access-gcl7q\") pod \"maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:05.229727 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.229703    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57d3d212-b9ce-40da-ac77-367f135fce62-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:05.231608 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.231586    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57d3d212-b9ce-40da-ac77-367f135fce62-maas-api-tls\") pod \"maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:05.237542 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.237514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcl7q\" (UniqueName: \"kubernetes.io/projected/57d3d212-b9ce-40da-ac77-367f135fce62-kube-api-access-gcl7q\") pod \"maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:05.264545 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.264516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:05.407308 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.407280    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"]
Aug 04 21:27:05.409405 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:27:05.409376    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57d3d212_b9ce_40da_ac77_367f135fce62.slice/crio-d60a557400e92994a23648c580f8e6ae7b1fdfa415ebbe906b5be64fd231d3d4 WatchSource:0}: Error finding container d60a557400e92994a23648c580f8e6ae7b1fdfa415ebbe906b5be64fd231d3d4: Status 404 returned error can't find the container with id d60a557400e92994a23648c580f8e6ae7b1fdfa415ebbe906b5be64fd231d3d4
Aug 04 21:27:05.599498 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:05.599404    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn" event={"ID":"57d3d212-b9ce-40da-ac77-367f135fce62","Type":"ContainerStarted","Data":"d60a557400e92994a23648c580f8e6ae7b1fdfa415ebbe906b5be64fd231d3d4"}
Aug 04 21:27:06.606335 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:06.606280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn" event={"ID":"57d3d212-b9ce-40da-ac77-367f135fce62","Type":"ContainerStarted","Data":"40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008"}
Aug 04 21:27:06.606754 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:06.606365    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:27:06.627208 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:06.627156    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn" podStartSLOduration=2.225697786 podStartE2EDuration="2.627139897s" totalImagesPullingTime="401.442111ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:27:06.624184052 +0000 UTC m=+2773.867803334" watchObservedRunningTime="2026-08-04 21:27:06.627139897 +0000 UTC m=+2773.870759178"
Aug 04 21:27:12.617168 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:27:12.617137    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:31:11.060648 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.060620    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"]
Aug 04 21:31:11.064176 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.064156    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.066816 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.066792    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a189ec\"/\"openshift-service-ca.crt\""
Aug 04 21:31:11.067947 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.067923    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a189ec\"/\"sub-test-model-a189ec-kserve-self-signed-certs\""
Aug 04 21:31:11.067947 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.067923    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a189ec\"/\"default-dockercfg-8rzqr\""
Aug 04 21:31:11.067947 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.067926    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a189ec\"/\"kube-root-ca.crt\""
Aug 04 21:31:11.073694 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.073675    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"]
Aug 04 21:31:11.132187 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.132154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-tmp-dir\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.132365 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.132192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-dshm\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.132365 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.132223    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-home\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.132365 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.132292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de452518-952d-4fdb-8add-67bc35f67fbf-tls-certs\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.132365 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.132363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-model-cache\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.132542 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.132382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvj84\" (UniqueName: \"kubernetes.io/projected/de452518-952d-4fdb-8add-67bc35f67fbf-kube-api-access-lvj84\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.232776 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.232749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de452518-952d-4fdb-8add-67bc35f67fbf-tls-certs\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.232896 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.232796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-model-cache\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.232896 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.232822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvj84\" (UniqueName: \"kubernetes.io/projected/de452518-952d-4fdb-8add-67bc35f67fbf-kube-api-access-lvj84\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.232896 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.232862    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-tmp-dir\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.233051 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.232897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-dshm\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.233051 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.232931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-home\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.233217 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.233190    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-model-cache\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.233292 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.233246    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-tmp-dir\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.233292 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.233265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-home\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.235137 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.235119    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-dshm\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.235437 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.235421    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de452518-952d-4fdb-8add-67bc35f67fbf-tls-certs\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.241482 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.241459    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvj84\" (UniqueName: \"kubernetes.io/projected/de452518-952d-4fdb-8add-67bc35f67fbf-kube-api-access-lvj84\") pod \"sub-test-model-a189ec-kserve-7c5b88f64f-q68kk\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") " pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.373915 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.373895    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:11.513400 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.513372    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"]
Aug 04 21:31:11.514677 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:31:11.514655    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde452518_952d_4fdb_8add_67bc35f67fbf.slice/crio-d6bdf985dbb35131de5dfe159c11d93c445c7cf49ea53ecaed665e1cbe4eb635 WatchSource:0}: Error finding container d6bdf985dbb35131de5dfe159c11d93c445c7cf49ea53ecaed665e1cbe4eb635: Status 404 returned error can't find the container with id d6bdf985dbb35131de5dfe159c11d93c445c7cf49ea53ecaed665e1cbe4eb635
Aug 04 21:31:11.937698 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.937662    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk" event={"ID":"de452518-952d-4fdb-8add-67bc35f67fbf","Type":"ContainerStarted","Data":"49c6c5010ed81a1729935de08429711fe0a816ae18cb75e5983eeb836ae09c08"}
Aug 04 21:31:11.937864 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.937704    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk" event={"ID":"de452518-952d-4fdb-8add-67bc35f67fbf","Type":"ContainerStarted","Data":"d6bdf985dbb35131de5dfe159c11d93c445c7cf49ea53ecaed665e1cbe4eb635"}
Aug 04 21:31:11.955562 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:11.955514    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk" podStartSLOduration=0.955499052 podStartE2EDuration="955.499052ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:31:11.95406586 +0000 UTC m=+3019.197685144" watchObservedRunningTime="2026-08-04 21:31:11.955499052 +0000 UTC m=+3019.199118333"
Aug 04 21:31:21.374635 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:21.374592    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:21.374635 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:21.374642    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:21.387143 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:21.387123    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:31:21.999735 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:31:21.999709    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:32:56.405373 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:32:56.405340    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"]
Aug 04 21:33:11.424297 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.424262    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk" podUID="de452518-952d-4fdb-8add-67bc35f67fbf" containerName="main" containerID="cri-o://49c6c5010ed81a1729935de08429711fe0a816ae18cb75e5983eeb836ae09c08" gracePeriod=45
Aug 04 21:33:11.588552 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.588513    2563 generic.go:356] "Generic (PLEG): container finished" podID="de452518-952d-4fdb-8add-67bc35f67fbf" containerID="49c6c5010ed81a1729935de08429711fe0a816ae18cb75e5983eeb836ae09c08" exitCode=0
Aug 04 21:33:11.588700 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.588579    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk" event={"ID":"de452518-952d-4fdb-8add-67bc35f67fbf","Type":"ContainerDied","Data":"49c6c5010ed81a1729935de08429711fe0a816ae18cb75e5983eeb836ae09c08"}
Aug 04 21:33:11.677920 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.677865    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:33:11.756894 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.756868    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-dshm\") pod \"de452518-952d-4fdb-8add-67bc35f67fbf\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") "
Aug 04 21:33:11.757025 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.756940    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-tmp-dir\") pod \"de452518-952d-4fdb-8add-67bc35f67fbf\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") "
Aug 04 21:33:11.757025 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.756955    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-home\") pod \"de452518-952d-4fdb-8add-67bc35f67fbf\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") "
Aug 04 21:33:11.757025 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.756970    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de452518-952d-4fdb-8add-67bc35f67fbf-tls-certs\") pod \"de452518-952d-4fdb-8add-67bc35f67fbf\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") "
Aug 04 21:33:11.757025 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.756991    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvj84\" (UniqueName: \"kubernetes.io/projected/de452518-952d-4fdb-8add-67bc35f67fbf-kube-api-access-lvj84\") pod \"de452518-952d-4fdb-8add-67bc35f67fbf\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") "
Aug 04 21:33:11.757025 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.757011    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-model-cache\") pod \"de452518-952d-4fdb-8add-67bc35f67fbf\" (UID: \"de452518-952d-4fdb-8add-67bc35f67fbf\") "
Aug 04 21:33:11.757275 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.757223    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-home" (OuterVolumeSpecName: "home") pod "de452518-952d-4fdb-8add-67bc35f67fbf" (UID: "de452518-952d-4fdb-8add-67bc35f67fbf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 21:33:11.757398 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.757368    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de452518-952d-4fdb-8add-67bc35f67fbf" (UID: "de452518-952d-4fdb-8add-67bc35f67fbf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 21:33:11.757518 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.757398    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-model-cache" (OuterVolumeSpecName: "model-cache") pod "de452518-952d-4fdb-8add-67bc35f67fbf" (UID: "de452518-952d-4fdb-8add-67bc35f67fbf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 21:33:11.759096 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.759068    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-dshm" (OuterVolumeSpecName: "dshm") pod "de452518-952d-4fdb-8add-67bc35f67fbf" (UID: "de452518-952d-4fdb-8add-67bc35f67fbf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 21:33:11.759181 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.759160    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de452518-952d-4fdb-8add-67bc35f67fbf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de452518-952d-4fdb-8add-67bc35f67fbf" (UID: "de452518-952d-4fdb-8add-67bc35f67fbf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:33:11.759181 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.759167    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de452518-952d-4fdb-8add-67bc35f67fbf-kube-api-access-lvj84" (OuterVolumeSpecName: "kube-api-access-lvj84") pod "de452518-952d-4fdb-8add-67bc35f67fbf" (UID: "de452518-952d-4fdb-8add-67bc35f67fbf"). InnerVolumeSpecName "kube-api-access-lvj84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:33:11.858011 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.857984    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-tmp-dir\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:33:11.858011 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.858008    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-home\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:33:11.858011 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.858016    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de452518-952d-4fdb-8add-67bc35f67fbf-tls-certs\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:33:11.858174 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.858024    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvj84\" (UniqueName: \"kubernetes.io/projected/de452518-952d-4fdb-8add-67bc35f67fbf-kube-api-access-lvj84\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:33:11.858174 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.858033    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-model-cache\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:33:11.858174 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:11.858040    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de452518-952d-4fdb-8add-67bc35f67fbf-dshm\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:33:12.596840 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:12.596803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk" event={"ID":"de452518-952d-4fdb-8add-67bc35f67fbf","Type":"ContainerDied","Data":"d6bdf985dbb35131de5dfe159c11d93c445c7cf49ea53ecaed665e1cbe4eb635"}
Aug 04 21:33:12.597298 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:12.596858    2563 scope.go:117] "RemoveContainer" containerID="49c6c5010ed81a1729935de08429711fe0a816ae18cb75e5983eeb836ae09c08"
Aug 04 21:33:12.597298 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:12.596816    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"
Aug 04 21:33:12.620332 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:12.620298    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"]
Aug 04 21:33:12.624495 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:12.624473    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a189ec/sub-test-model-a189ec-kserve-7c5b88f64f-q68kk"]
Aug 04 21:33:13.243740 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:13.243712    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de452518-952d-4fdb-8add-67bc35f67fbf" path="/var/lib/kubelet/pods/de452518-952d-4fdb-8add-67bc35f67fbf/volumes"
Aug 04 21:33:28.858028 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.857952    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"]
Aug 04 21:33:28.858594 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.858575    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de452518-952d-4fdb-8add-67bc35f67fbf" containerName="main"
Aug 04 21:33:28.858639 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.858599    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de452518-952d-4fdb-8add-67bc35f67fbf" containerName="main"
Aug 04 21:33:28.858700 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.858689    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="de452518-952d-4fdb-8add-67bc35f67fbf" containerName="main"
Aug 04 21:33:28.863452 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.863429    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:28.866252 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.866232    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cd644a\"/\"kube-root-ca.crt\""
Aug 04 21:33:28.866388 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.866251    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cd644a\"/\"openshift-service-ca.crt\""
Aug 04 21:33:28.867268 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.867250    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cd644a\"/\"rate-test-model-cd644a-kserve-self-signed-certs\""
Aug 04 21:33:28.867396 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.867296    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cd644a\"/\"default-dockercfg-ngjrk\""
Aug 04 21:33:28.871751 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.871734    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"]
Aug 04 21:33:28.908639 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.908600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-dshm\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:28.908786 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.908643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-tmp-dir\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:28.908786 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.908718    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-model-cache\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:28.908786 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.908763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce69d23f-4156-43e3-95b5-d60a48c3793c-tls-certs\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:28.908948 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.908814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-home\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:28.908948 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:28.908856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb5tm\" (UniqueName: \"kubernetes.io/projected/ce69d23f-4156-43e3-95b5-d60a48c3793c-kube-api-access-gb5tm\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.009713 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.009691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-model-cache\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.009851 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.009720    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce69d23f-4156-43e3-95b5-d60a48c3793c-tls-certs\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.009851 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.009743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-home\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.009851 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.009775    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb5tm\" (UniqueName: \"kubernetes.io/projected/ce69d23f-4156-43e3-95b5-d60a48c3793c-kube-api-access-gb5tm\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.009851 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.009815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-dshm\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.010062 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.009855    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-tmp-dir\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.010159 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.010135    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-model-cache\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.010227 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.010182    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-home\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.010227 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.010203    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-tmp-dir\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.012197 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.012169    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-dshm\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.012453 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.012437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce69d23f-4156-43e3-95b5-d60a48c3793c-tls-certs\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.018856 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.018836    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb5tm\" (UniqueName: \"kubernetes.io/projected/ce69d23f-4156-43e3-95b5-d60a48c3793c-kube-api-access-gb5tm\") pod \"rate-test-model-cd644a-kserve-6869b4587-qgk7x\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") " pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.173558 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.173531    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:29.320597 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.320570    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"]
Aug 04 21:33:29.322122 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:33:29.322095    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce69d23f_4156_43e3_95b5_d60a48c3793c.slice/crio-4b44a199dad3c20f6d2e662f99d2f7466ca44fd3a642846d8a0c7cc9a114cb5f WatchSource:0}: Error finding container 4b44a199dad3c20f6d2e662f99d2f7466ca44fd3a642846d8a0c7cc9a114cb5f: Status 404 returned error can't find the container with id 4b44a199dad3c20f6d2e662f99d2f7466ca44fd3a642846d8a0c7cc9a114cb5f
Aug 04 21:33:29.690427 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.690385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x" event={"ID":"ce69d23f-4156-43e3-95b5-d60a48c3793c","Type":"ContainerStarted","Data":"64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8"}
Aug 04 21:33:29.690629 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.690441    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x" event={"ID":"ce69d23f-4156-43e3-95b5-d60a48c3793c","Type":"ContainerStarted","Data":"4b44a199dad3c20f6d2e662f99d2f7466ca44fd3a642846d8a0c7cc9a114cb5f"}
Aug 04 21:33:29.714685 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:29.714640    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x" podStartSLOduration=1.714624772 podStartE2EDuration="1.714624772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:33:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:33:29.712568732 +0000 UTC m=+3156.956188014" watchObservedRunningTime="2026-08-04 21:33:29.714624772 +0000 UTC m=+3156.958244054"
Aug 04 21:33:39.174450 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:39.174406    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:39.174868 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:39.174494    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:39.186628 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:39.186603    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:33:39.762457 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:33:39.762428    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:34:40.447491 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:40.447461    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"]
Aug 04 21:34:55.466505 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.466393    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x" podUID="ce69d23f-4156-43e3-95b5-d60a48c3793c" containerName="main" containerID="cri-o://64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8" gracePeriod=45
Aug 04 21:34:55.706240 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.706218    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:34:55.776680 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.776618    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-model-cache\") pod \"ce69d23f-4156-43e3-95b5-d60a48c3793c\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") "
Aug 04 21:34:55.776680 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.776647    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-home\") pod \"ce69d23f-4156-43e3-95b5-d60a48c3793c\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") "
Aug 04 21:34:55.776877 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.776700    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb5tm\" (UniqueName: \"kubernetes.io/projected/ce69d23f-4156-43e3-95b5-d60a48c3793c-kube-api-access-gb5tm\") pod \"ce69d23f-4156-43e3-95b5-d60a48c3793c\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") "
Aug 04 21:34:55.776877 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.776738    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-tmp-dir\") pod \"ce69d23f-4156-43e3-95b5-d60a48c3793c\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") "
Aug 04 21:34:55.776877 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.776758    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-dshm\") pod \"ce69d23f-4156-43e3-95b5-d60a48c3793c\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") "
Aug 04 21:34:55.776877 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.776787    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce69d23f-4156-43e3-95b5-d60a48c3793c-tls-certs\") pod \"ce69d23f-4156-43e3-95b5-d60a48c3793c\" (UID: \"ce69d23f-4156-43e3-95b5-d60a48c3793c\") "
Aug 04 21:34:55.777078 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.776921    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce69d23f-4156-43e3-95b5-d60a48c3793c" (UID: "ce69d23f-4156-43e3-95b5-d60a48c3793c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 21:34:55.777078 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.776979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce69d23f-4156-43e3-95b5-d60a48c3793c" (UID: "ce69d23f-4156-43e3-95b5-d60a48c3793c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 21:34:55.777078 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.777032    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-home" (OuterVolumeSpecName: "home") pod "ce69d23f-4156-43e3-95b5-d60a48c3793c" (UID: "ce69d23f-4156-43e3-95b5-d60a48c3793c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 21:34:55.777078 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.777047    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-model-cache\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:34:55.777078 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.777065    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-tmp-dir\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:34:55.779165 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.779132    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce69d23f-4156-43e3-95b5-d60a48c3793c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce69d23f-4156-43e3-95b5-d60a48c3793c" (UID: "ce69d23f-4156-43e3-95b5-d60a48c3793c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:34:55.779165 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.779136    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-dshm" (OuterVolumeSpecName: "dshm") pod "ce69d23f-4156-43e3-95b5-d60a48c3793c" (UID: "ce69d23f-4156-43e3-95b5-d60a48c3793c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 21:34:55.779349 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.779163    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce69d23f-4156-43e3-95b5-d60a48c3793c-kube-api-access-gb5tm" (OuterVolumeSpecName: "kube-api-access-gb5tm") pod "ce69d23f-4156-43e3-95b5-d60a48c3793c" (UID: "ce69d23f-4156-43e3-95b5-d60a48c3793c"). InnerVolumeSpecName "kube-api-access-gb5tm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:34:55.878307 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.878285    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-home\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:34:55.878307 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.878305    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb5tm\" (UniqueName: \"kubernetes.io/projected/ce69d23f-4156-43e3-95b5-d60a48c3793c-kube-api-access-gb5tm\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:34:55.878470 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.878338    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce69d23f-4156-43e3-95b5-d60a48c3793c-dshm\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:34:55.878470 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:55.878347    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce69d23f-4156-43e3-95b5-d60a48c3793c-tls-certs\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:34:56.179185 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:56.179153    2563 generic.go:356] "Generic (PLEG): container finished" podID="ce69d23f-4156-43e3-95b5-d60a48c3793c" containerID="64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8" exitCode=0
Aug 04 21:34:56.179390 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:56.179199    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"
Aug 04 21:34:56.179390 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:56.179214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x" event={"ID":"ce69d23f-4156-43e3-95b5-d60a48c3793c","Type":"ContainerDied","Data":"64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8"}
Aug 04 21:34:56.179390 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:56.179253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x" event={"ID":"ce69d23f-4156-43e3-95b5-d60a48c3793c","Type":"ContainerDied","Data":"4b44a199dad3c20f6d2e662f99d2f7466ca44fd3a642846d8a0c7cc9a114cb5f"}
Aug 04 21:34:56.179390 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:56.179273    2563 scope.go:117] "RemoveContainer" containerID="64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8"
Aug 04 21:34:56.188783 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:56.188767    2563 scope.go:117] "RemoveContainer" containerID="64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8"
Aug 04 21:34:56.189017 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:34:56.188995    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8\": container with ID starting with 64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8 not found: ID does not exist" containerID="64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8"
Aug 04 21:34:56.189083 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:56.189026    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8"} err="failed to get container status \"64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8\": rpc error: code = NotFound desc = could not find container \"64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8\": container with ID starting with 64e89be477f27d8557b740ea02e99a19da0be07bf9f00eb7718bc91326096dd8 not found: ID does not exist"
Aug 04 21:34:56.201890 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:56.201871    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"]
Aug 04 21:34:56.205927 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:56.205908    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-cd644a/rate-test-model-cd644a-kserve-6869b4587-qgk7x"]
Aug 04 21:34:57.243773 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:34:57.243743    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce69d23f-4156-43e3-95b5-d60a48c3793c" path="/var/lib/kubelet/pods/ce69d23f-4156-43e3-95b5-d60a48c3793c/volumes"
Aug 04 21:35:04.195816 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.195784    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"]
Aug 04 21:35:04.196552 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.196454    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce69d23f-4156-43e3-95b5-d60a48c3793c" containerName="main"
Aug 04 21:35:04.196552 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.196475    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce69d23f-4156-43e3-95b5-d60a48c3793c" containerName="main"
Aug 04 21:35:04.196861 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.196676    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce69d23f-4156-43e3-95b5-d60a48c3793c" containerName="main"
Aug 04 21:35:04.201436 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.201413    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:04.203908 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.203889    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-e783f6-serving-cert\""
Aug 04 21:35:04.209880 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.209857    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"]
Aug 04 21:35:04.244965 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.244938    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fed576e8-991f-41d7-8615-76ebb86a0e8d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:04.245099 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.244989    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed576e8-991f-41d7-8615-76ebb86a0e8d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:04.245099 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.245061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45jv2\" (UniqueName: \"kubernetes.io/projected/fed576e8-991f-41d7-8615-76ebb86a0e8d-kube-api-access-45jv2\") pod \"maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:04.346091 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.346064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed576e8-991f-41d7-8615-76ebb86a0e8d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:04.346223 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.346106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45jv2\" (UniqueName: \"kubernetes.io/projected/fed576e8-991f-41d7-8615-76ebb86a0e8d-kube-api-access-45jv2\") pod \"maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:04.346223 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.346165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fed576e8-991f-41d7-8615-76ebb86a0e8d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:04.346296 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:35:04.346272    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-serving-cert: secret "maas-api-e2e-ipp-a-e783f6-serving-cert" not found
Aug 04 21:35:04.346368 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:35:04.346362    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fed576e8-991f-41d7-8615-76ebb86a0e8d-maas-api-tls podName:fed576e8-991f-41d7-8615-76ebb86a0e8d nodeName:}" failed. No retries permitted until 2026-08-04 21:35:04.846310891 +0000 UTC m=+3252.089930153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fed576e8-991f-41d7-8615-76ebb86a0e8d-maas-api-tls") pod "maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv" (UID: "fed576e8-991f-41d7-8615-76ebb86a0e8d") : secret "maas-api-e2e-ipp-a-e783f6-serving-cert" not found
Aug 04 21:35:04.346796 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.346777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed576e8-991f-41d7-8615-76ebb86a0e8d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:04.357651 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.357628    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45jv2\" (UniqueName: \"kubernetes.io/projected/fed576e8-991f-41d7-8615-76ebb86a0e8d-kube-api-access-45jv2\") pod \"maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:04.849646 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.849609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fed576e8-991f-41d7-8615-76ebb86a0e8d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:04.852133 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:04.852108    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fed576e8-991f-41d7-8615-76ebb86a0e8d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:05.113771 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:05.113692    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:05.259837 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:05.259806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"]
Aug 04 21:35:05.267154 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:35:05.267131    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed576e8_991f_41d7_8615_76ebb86a0e8d.slice/crio-7de44b7818c38f206453d98b6cac6c9c8ecbcf244c81349cc557c502080ec066 WatchSource:0}: Error finding container 7de44b7818c38f206453d98b6cac6c9c8ecbcf244c81349cc557c502080ec066: Status 404 returned error can't find the container with id 7de44b7818c38f206453d98b6cac6c9c8ecbcf244c81349cc557c502080ec066
Aug 04 21:35:05.272086 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:05.272068    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 21:35:06.235219 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:06.235186    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv" event={"ID":"fed576e8-991f-41d7-8615-76ebb86a0e8d","Type":"ContainerStarted","Data":"7de44b7818c38f206453d98b6cac6c9c8ecbcf244c81349cc557c502080ec066"}
Aug 04 21:35:07.244094 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:07.244069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:07.244473 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:07.244116    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv" event={"ID":"fed576e8-991f-41d7-8615-76ebb86a0e8d","Type":"ContainerStarted","Data":"30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee"}
Aug 04 21:35:07.265831 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:07.265789    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv" podStartSLOduration=1.395933536 podStartE2EDuration="3.265775066s" totalImagesPullingTime="1.86984153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:35:07.264081882 +0000 UTC m=+3254.507701163" watchObservedRunningTime="2026-08-04 21:35:07.265775066 +0000 UTC m=+3254.509394347"
Aug 04 21:35:13.253502 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:13.253471    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:35:33.317339 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.317295    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"]
Aug 04 21:35:33.321122 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.321100    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:33.324111 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.324087    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-80be56-serving-cert\""
Aug 04 21:35:33.335652 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.335622    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"]
Aug 04 21:35:33.482458 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.482426    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b03620ff-baed-43b5-8d82-15349d8020eb-maas-api-tls\") pod \"maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:33.482458 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.482455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b03620ff-baed-43b5-8d82-15349d8020eb-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:33.482657 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.482613    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdwqt\" (UniqueName: \"kubernetes.io/projected/b03620ff-baed-43b5-8d82-15349d8020eb-kube-api-access-gdwqt\") pod \"maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:33.583175 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.583097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdwqt\" (UniqueName: \"kubernetes.io/projected/b03620ff-baed-43b5-8d82-15349d8020eb-kube-api-access-gdwqt\") pod \"maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:33.583175 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.583132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b03620ff-baed-43b5-8d82-15349d8020eb-maas-api-tls\") pod \"maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:33.583175 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.583151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b03620ff-baed-43b5-8d82-15349d8020eb-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:33.583790 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.583772    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b03620ff-baed-43b5-8d82-15349d8020eb-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:33.585567 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.585551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b03620ff-baed-43b5-8d82-15349d8020eb-maas-api-tls\") pod \"maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:33.591501 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.591477    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdwqt\" (UniqueName: \"kubernetes.io/projected/b03620ff-baed-43b5-8d82-15349d8020eb-kube-api-access-gdwqt\") pod \"maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:33.630967 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.630944    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:33.983029 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:33.982984    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"]
Aug 04 21:35:33.985090 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:35:33.985062    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb03620ff_baed_43b5_8d82_15349d8020eb.slice/crio-8485a6815796b146240daa6e5f397b32fa4c9b0537fbcb7be6bf0864c52ac611 WatchSource:0}: Error finding container 8485a6815796b146240daa6e5f397b32fa4c9b0537fbcb7be6bf0864c52ac611: Status 404 returned error can't find the container with id 8485a6815796b146240daa6e5f397b32fa4c9b0537fbcb7be6bf0864c52ac611
Aug 04 21:35:34.122694 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.122666    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"]
Aug 04 21:35:34.127257 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.127240    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:35:34.129779 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.129760    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-80be56\""
Aug 04 21:35:34.129918 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.129899    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-80be56-dockercfg-r25nh\""
Aug 04 21:35:34.138105 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.138086    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"]
Aug 04 21:35:34.289756 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.289684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpz2s\" (UniqueName: \"kubernetes.io/projected/99c04223-3aed-47f8-adac-ea1a48416d03-kube-api-access-jpz2s\") pod \"payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn\" (UID: \"99c04223-3aed-47f8-adac-ea1a48416d03\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:35:34.289756 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.289724    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99c04223-3aed-47f8-adac-ea1a48416d03-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn\" (UID: \"99c04223-3aed-47f8-adac-ea1a48416d03\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:35:34.386121 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.386089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4" event={"ID":"b03620ff-baed-43b5-8d82-15349d8020eb","Type":"ContainerStarted","Data":"8485a6815796b146240daa6e5f397b32fa4c9b0537fbcb7be6bf0864c52ac611"}
Aug 04 21:35:34.391067 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.391028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpz2s\" (UniqueName: \"kubernetes.io/projected/99c04223-3aed-47f8-adac-ea1a48416d03-kube-api-access-jpz2s\") pod \"payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn\" (UID: \"99c04223-3aed-47f8-adac-ea1a48416d03\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:35:34.391218 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.391086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99c04223-3aed-47f8-adac-ea1a48416d03-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn\" (UID: \"99c04223-3aed-47f8-adac-ea1a48416d03\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:35:34.392132 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.392103    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99c04223-3aed-47f8-adac-ea1a48416d03-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn\" (UID: \"99c04223-3aed-47f8-adac-ea1a48416d03\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:35:34.401738 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.401708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpz2s\" (UniqueName: \"kubernetes.io/projected/99c04223-3aed-47f8-adac-ea1a48416d03-kube-api-access-jpz2s\") pod \"payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn\" (UID: \"99c04223-3aed-47f8-adac-ea1a48416d03\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:35:34.439716 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.439693    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:35:34.577446 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:34.577421    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"]
Aug 04 21:35:34.578962 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:35:34.578934    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c04223_3aed_47f8_adac_ea1a48416d03.slice/crio-9e581f5d2c0d17ae04eecb81b9091f9a30c512885bf0aab3836efceba1523bf4 WatchSource:0}: Error finding container 9e581f5d2c0d17ae04eecb81b9091f9a30c512885bf0aab3836efceba1523bf4: Status 404 returned error can't find the container with id 9e581f5d2c0d17ae04eecb81b9091f9a30c512885bf0aab3836efceba1523bf4
Aug 04 21:35:35.393522 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:35.393481    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn" event={"ID":"99c04223-3aed-47f8-adac-ea1a48416d03","Type":"ContainerStarted","Data":"7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a"}
Aug 04 21:35:35.394013 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:35.393549    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:35:35.394013 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:35.393570    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn" event={"ID":"99c04223-3aed-47f8-adac-ea1a48416d03","Type":"ContainerStarted","Data":"9e581f5d2c0d17ae04eecb81b9091f9a30c512885bf0aab3836efceba1523bf4"}
Aug 04 21:35:35.395276 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:35.395251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4" event={"ID":"b03620ff-baed-43b5-8d82-15349d8020eb","Type":"ContainerStarted","Data":"749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710"}
Aug 04 21:35:35.395464 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:35.395306    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:35.412151 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:35.412111    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn" podStartSLOduration=1.412094591 podStartE2EDuration="1.412094591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:35:35.4111143 +0000 UTC m=+3282.654733583" watchObservedRunningTime="2026-08-04 21:35:35.412094591 +0000 UTC m=+3282.655713872"
Aug 04 21:35:35.435096 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:35.435049    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4" podStartSLOduration=1.808359557 podStartE2EDuration="2.435037291s" totalImagesPullingTime="626.677734ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:35:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:35:35.43212087 +0000 UTC m=+3282.675740153" watchObservedRunningTime="2026-08-04 21:35:35.435037291 +0000 UTC m=+3282.678656572"
Aug 04 21:35:41.406905 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:41.406876    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:35:46.403565 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:35:46.403305    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:36:04.330984 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:36:04.330957    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn_99c04223-3aed-47f8-adac-ea1a48416d03/payload-processing/0.log"
Aug 04 21:36:54.576671 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:36:54.576642    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn_99c04223-3aed-47f8-adac-ea1a48416d03/payload-processing/0.log"
Aug 04 21:37:00.514583 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.514549    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"]
Aug 04 21:37:00.519228 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.519202    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:00.521743 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.521718    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-354c63-serving-cert\""
Aug 04 21:37:00.531742 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.531712    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"]
Aug 04 21:37:00.632482 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.632457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:00.632638 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.632495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7882\" (UniqueName: \"kubernetes.io/projected/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-kube-api-access-t7882\") pod \"maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:00.632638 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.632542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:00.733296 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.733269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:00.733427 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.733297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7882\" (UniqueName: \"kubernetes.io/projected/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-kube-api-access-t7882\") pod \"maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:00.733427 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.733382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:00.733427 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:37:00.733414    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-serving-cert: secret "maas-api-e2e-ipp-cleanup-354c63-serving-cert" not found
Aug 04 21:37:00.733612 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:37:00.733483    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-maas-api-tls podName:fe7e60b0-1657-4fea-8532-5f3a1d5e03f2 nodeName:}" failed. No retries permitted until 2026-08-04 21:37:01.233463728 +0000 UTC m=+3368.477082990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd" (UID: "fe7e60b0-1657-4fea-8532-5f3a1d5e03f2") : secret "maas-api-e2e-ipp-cleanup-354c63-serving-cert" not found
Aug 04 21:37:00.734014 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.733992    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:00.749260 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:00.749241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7882\" (UniqueName: \"kubernetes.io/projected/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-kube-api-access-t7882\") pod \"maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:01.237593 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:01.237557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:01.240128 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:01.240103    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:01.434875 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:01.434848    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:01.578919 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:01.578893    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"]
Aug 04 21:37:01.580633 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:37:01.580602    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe7e60b0_1657_4fea_8532_5f3a1d5e03f2.slice/crio-cb746dc6630d5f6aad1e3f356fa6f435d4f81a77546be19f730fee9a2240b990 WatchSource:0}: Error finding container cb746dc6630d5f6aad1e3f356fa6f435d4f81a77546be19f730fee9a2240b990: Status 404 returned error can't find the container with id cb746dc6630d5f6aad1e3f356fa6f435d4f81a77546be19f730fee9a2240b990
Aug 04 21:37:01.876691 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:01.876658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd" event={"ID":"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2","Type":"ContainerStarted","Data":"cb746dc6630d5f6aad1e3f356fa6f435d4f81a77546be19f730fee9a2240b990"}
Aug 04 21:37:02.883608 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:02.883573    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd" event={"ID":"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2","Type":"ContainerStarted","Data":"a72cc493c5e34b9ec053cb4c153804fdfbb1cd46fdf38ac454fbc08240a02f29"}
Aug 04 21:37:02.884009 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:02.883704    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:02.905431 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:02.905391    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd" podStartSLOduration=2.436714245 podStartE2EDuration="2.905377385s" totalImagesPullingTime="468.66314ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 21:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:37:02.90371528 +0000 UTC m=+3370.147334561" watchObservedRunningTime="2026-08-04 21:37:02.905377385 +0000 UTC m=+3370.148996668"
Aug 04 21:37:08.895214 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:08.895180    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:21.862794 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:21.862753    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"]
Aug 04 21:37:21.863361 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:21.863007    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd" podUID="fe7e60b0-1657-4fea-8532-5f3a1d5e03f2" containerName="maas-api" containerID="cri-o://a72cc493c5e34b9ec053cb4c153804fdfbb1cd46fdf38ac454fbc08240a02f29" gracePeriod=30
Aug 04 21:37:21.997497 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:21.997456    2563 generic.go:356] "Generic (PLEG): container finished" podID="fe7e60b0-1657-4fea-8532-5f3a1d5e03f2" containerID="a72cc493c5e34b9ec053cb4c153804fdfbb1cd46fdf38ac454fbc08240a02f29" exitCode=0
Aug 04 21:37:21.997671 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:21.997539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd" event={"ID":"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2","Type":"ContainerDied","Data":"a72cc493c5e34b9ec053cb4c153804fdfbb1cd46fdf38ac454fbc08240a02f29"}
Aug 04 21:37:22.123760 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:22.123737    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:22.221872 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:22.221832    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7882\" (UniqueName: \"kubernetes.io/projected/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-kube-api-access-t7882\") pod \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") "
Aug 04 21:37:22.222152 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:22.221883    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-openshift-service-ca\") pod \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") "
Aug 04 21:37:22.222152 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:22.222026    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-maas-api-tls\") pod \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\" (UID: \"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2\") "
Aug 04 21:37:22.222428 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:22.222394    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fe7e60b0-1657-4fea-8532-5f3a1d5e03f2" (UID: "fe7e60b0-1657-4fea-8532-5f3a1d5e03f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:37:22.224280 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:22.224251    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-kube-api-access-t7882" (OuterVolumeSpecName: "kube-api-access-t7882") pod "fe7e60b0-1657-4fea-8532-5f3a1d5e03f2" (UID: "fe7e60b0-1657-4fea-8532-5f3a1d5e03f2"). InnerVolumeSpecName "kube-api-access-t7882". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:37:22.224550 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:22.224525    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fe7e60b0-1657-4fea-8532-5f3a1d5e03f2" (UID: "fe7e60b0-1657-4fea-8532-5f3a1d5e03f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:37:22.323103 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:22.323067    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7882\" (UniqueName: \"kubernetes.io/projected/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-kube-api-access-t7882\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:37:22.323103 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:22.323104    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:37:22.323349 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:22.323118    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:37:23.004405 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:23.004369    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"
Aug 04 21:37:23.004893 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:23.004446    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd" event={"ID":"fe7e60b0-1657-4fea-8532-5f3a1d5e03f2","Type":"ContainerDied","Data":"cb746dc6630d5f6aad1e3f356fa6f435d4f81a77546be19f730fee9a2240b990"}
Aug 04 21:37:23.004893 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:23.004519    2563 scope.go:117] "RemoveContainer" containerID="a72cc493c5e34b9ec053cb4c153804fdfbb1cd46fdf38ac454fbc08240a02f29"
Aug 04 21:37:23.026764 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:23.026741    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"]
Aug 04 21:37:23.031265 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:23.031247    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-354c63-597d485774-nmhmd"]
Aug 04 21:37:23.245276 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:23.245243    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe7e60b0-1657-4fea-8532-5f3a1d5e03f2" path="/var/lib/kubelet/pods/fe7e60b0-1657-4fea-8532-5f3a1d5e03f2/volumes"
Aug 04 21:37:51.795343 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:51.795229    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"]
Aug 04 21:37:51.795915 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:51.795463    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv" podUID="fed576e8-991f-41d7-8615-76ebb86a0e8d" containerName="maas-api" containerID="cri-o://30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee" gracePeriod=30
Aug 04 21:37:52.063413 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.063388    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:37:52.173511 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.173476    2563 generic.go:356] "Generic (PLEG): container finished" podID="fed576e8-991f-41d7-8615-76ebb86a0e8d" containerID="30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee" exitCode=0
Aug 04 21:37:52.173674 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.173522    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"
Aug 04 21:37:52.173674 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.173560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv" event={"ID":"fed576e8-991f-41d7-8615-76ebb86a0e8d","Type":"ContainerDied","Data":"30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee"}
Aug 04 21:37:52.173674 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.173630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv" event={"ID":"fed576e8-991f-41d7-8615-76ebb86a0e8d","Type":"ContainerDied","Data":"7de44b7818c38f206453d98b6cac6c9c8ecbcf244c81349cc557c502080ec066"}
Aug 04 21:37:52.173674 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.173665    2563 scope.go:117] "RemoveContainer" containerID="30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee"
Aug 04 21:37:52.182487 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.182465    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45jv2\" (UniqueName: \"kubernetes.io/projected/fed576e8-991f-41d7-8615-76ebb86a0e8d-kube-api-access-45jv2\") pod \"fed576e8-991f-41d7-8615-76ebb86a0e8d\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") "
Aug 04 21:37:52.182625 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.182500    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fed576e8-991f-41d7-8615-76ebb86a0e8d-maas-api-tls\") pod \"fed576e8-991f-41d7-8615-76ebb86a0e8d\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") "
Aug 04 21:37:52.182625 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.182544    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed576e8-991f-41d7-8615-76ebb86a0e8d-openshift-service-ca\") pod \"fed576e8-991f-41d7-8615-76ebb86a0e8d\" (UID: \"fed576e8-991f-41d7-8615-76ebb86a0e8d\") "
Aug 04 21:37:52.182906 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.182867    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fed576e8-991f-41d7-8615-76ebb86a0e8d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fed576e8-991f-41d7-8615-76ebb86a0e8d" (UID: "fed576e8-991f-41d7-8615-76ebb86a0e8d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:37:52.182995 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.182926    2563 scope.go:117] "RemoveContainer" containerID="30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee"
Aug 04 21:37:52.183378 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:37:52.183350    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee\": container with ID starting with 30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee not found: ID does not exist" containerID="30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee"
Aug 04 21:37:52.183465 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.183389    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee"} err="failed to get container status \"30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee\": rpc error: code = NotFound desc = could not find container \"30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee\": container with ID starting with 30c3cdfc5d573662a700e4fe8dba35610a9069074558c39ca5589531d1262eee not found: ID does not exist"
Aug 04 21:37:52.184652 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.184631    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fed576e8-991f-41d7-8615-76ebb86a0e8d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fed576e8-991f-41d7-8615-76ebb86a0e8d" (UID: "fed576e8-991f-41d7-8615-76ebb86a0e8d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:37:52.184765 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.184744    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed576e8-991f-41d7-8615-76ebb86a0e8d-kube-api-access-45jv2" (OuterVolumeSpecName: "kube-api-access-45jv2") pod "fed576e8-991f-41d7-8615-76ebb86a0e8d" (UID: "fed576e8-991f-41d7-8615-76ebb86a0e8d"). InnerVolumeSpecName "kube-api-access-45jv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:37:52.283421 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.283391    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45jv2\" (UniqueName: \"kubernetes.io/projected/fed576e8-991f-41d7-8615-76ebb86a0e8d-kube-api-access-45jv2\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:37:52.283421 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.283413    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fed576e8-991f-41d7-8615-76ebb86a0e8d-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:37:52.283421 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.283423    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed576e8-991f-41d7-8615-76ebb86a0e8d-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:37:52.495217 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.495192    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"]
Aug 04 21:37:52.498851 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:52.498827    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e783f6-5f548fb9b8-knpvv"]
Aug 04 21:37:53.244173 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:37:53.244144    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fed576e8-991f-41d7-8615-76ebb86a0e8d" path="/var/lib/kubelet/pods/fed576e8-991f-41d7-8615-76ebb86a0e8d/volumes"
Aug 04 21:38:48.833971 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:48.833937    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"]
Aug 04 21:38:48.834622 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:48.834160    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4" podUID="b03620ff-baed-43b5-8d82-15349d8020eb" containerName="maas-api" containerID="cri-o://749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710" gracePeriod=30
Aug 04 21:38:48.929262 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:48.929213    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"]
Aug 04 21:38:48.929506 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:48.929477    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn" podUID="99c04223-3aed-47f8-adac-ea1a48416d03" containerName="payload-processing" containerID="cri-o://7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a" gracePeriod=30
Aug 04 21:38:49.194164 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.194137    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:38:49.230085 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.230055    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:38:49.299415 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.299385    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b03620ff-baed-43b5-8d82-15349d8020eb-openshift-service-ca\") pod \"b03620ff-baed-43b5-8d82-15349d8020eb\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") "
Aug 04 21:38:49.299587 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.299434    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdwqt\" (UniqueName: \"kubernetes.io/projected/b03620ff-baed-43b5-8d82-15349d8020eb-kube-api-access-gdwqt\") pod \"b03620ff-baed-43b5-8d82-15349d8020eb\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") "
Aug 04 21:38:49.299587 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.299473    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b03620ff-baed-43b5-8d82-15349d8020eb-maas-api-tls\") pod \"b03620ff-baed-43b5-8d82-15349d8020eb\" (UID: \"b03620ff-baed-43b5-8d82-15349d8020eb\") "
Aug 04 21:38:49.299587 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.299546    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpz2s\" (UniqueName: \"kubernetes.io/projected/99c04223-3aed-47f8-adac-ea1a48416d03-kube-api-access-jpz2s\") pod \"99c04223-3aed-47f8-adac-ea1a48416d03\" (UID: \"99c04223-3aed-47f8-adac-ea1a48416d03\") "
Aug 04 21:38:49.299765 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.299709    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99c04223-3aed-47f8-adac-ea1a48416d03-plugins-config-volume\") pod \"99c04223-3aed-47f8-adac-ea1a48416d03\" (UID: \"99c04223-3aed-47f8-adac-ea1a48416d03\") "
Aug 04 21:38:49.299819 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.299784    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b03620ff-baed-43b5-8d82-15349d8020eb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b03620ff-baed-43b5-8d82-15349d8020eb" (UID: "b03620ff-baed-43b5-8d82-15349d8020eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:38:49.300072 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.300042    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b03620ff-baed-43b5-8d82-15349d8020eb-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:38:49.300231 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.300105    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99c04223-3aed-47f8-adac-ea1a48416d03-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "99c04223-3aed-47f8-adac-ea1a48416d03" (UID: "99c04223-3aed-47f8-adac-ea1a48416d03"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:38:49.301784 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.301760    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b03620ff-baed-43b5-8d82-15349d8020eb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b03620ff-baed-43b5-8d82-15349d8020eb" (UID: "b03620ff-baed-43b5-8d82-15349d8020eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:38:49.302199 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.302181    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b03620ff-baed-43b5-8d82-15349d8020eb-kube-api-access-gdwqt" (OuterVolumeSpecName: "kube-api-access-gdwqt") pod "b03620ff-baed-43b5-8d82-15349d8020eb" (UID: "b03620ff-baed-43b5-8d82-15349d8020eb"). InnerVolumeSpecName "kube-api-access-gdwqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:38:49.302280 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.302233    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99c04223-3aed-47f8-adac-ea1a48416d03-kube-api-access-jpz2s" (OuterVolumeSpecName: "kube-api-access-jpz2s") pod "99c04223-3aed-47f8-adac-ea1a48416d03" (UID: "99c04223-3aed-47f8-adac-ea1a48416d03"). InnerVolumeSpecName "kube-api-access-jpz2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:38:49.401163 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.401132    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpz2s\" (UniqueName: \"kubernetes.io/projected/99c04223-3aed-47f8-adac-ea1a48416d03-kube-api-access-jpz2s\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:38:49.401163 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.401161    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99c04223-3aed-47f8-adac-ea1a48416d03-plugins-config-volume\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:38:49.401374 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.401175    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdwqt\" (UniqueName: \"kubernetes.io/projected/b03620ff-baed-43b5-8d82-15349d8020eb-kube-api-access-gdwqt\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:38:49.401374 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.401190    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b03620ff-baed-43b5-8d82-15349d8020eb-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:38:49.493297 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.493258    2563 generic.go:356] "Generic (PLEG): container finished" podID="99c04223-3aed-47f8-adac-ea1a48416d03" containerID="7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a" exitCode=0
Aug 04 21:38:49.493501 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.493342    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"
Aug 04 21:38:49.493501 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.493392    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn" event={"ID":"99c04223-3aed-47f8-adac-ea1a48416d03","Type":"ContainerDied","Data":"7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a"}
Aug 04 21:38:49.493501 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.493458    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn" event={"ID":"99c04223-3aed-47f8-adac-ea1a48416d03","Type":"ContainerDied","Data":"9e581f5d2c0d17ae04eecb81b9091f9a30c512885bf0aab3836efceba1523bf4"}
Aug 04 21:38:49.493501 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.493488    2563 scope.go:117] "RemoveContainer" containerID="7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a"
Aug 04 21:38:49.495396 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.495368    2563 generic.go:356] "Generic (PLEG): container finished" podID="b03620ff-baed-43b5-8d82-15349d8020eb" containerID="749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710" exitCode=0
Aug 04 21:38:49.495498 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.495466    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"
Aug 04 21:38:49.495573 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.495500    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4" event={"ID":"b03620ff-baed-43b5-8d82-15349d8020eb","Type":"ContainerDied","Data":"749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710"}
Aug 04 21:38:49.495638 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.495574    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4" event={"ID":"b03620ff-baed-43b5-8d82-15349d8020eb","Type":"ContainerDied","Data":"8485a6815796b146240daa6e5f397b32fa4c9b0537fbcb7be6bf0864c52ac611"}
Aug 04 21:38:49.503803 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.503787    2563 scope.go:117] "RemoveContainer" containerID="7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a"
Aug 04 21:38:49.504064 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:38:49.504039    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a\": container with ID starting with 7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a not found: ID does not exist" containerID="7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a"
Aug 04 21:38:49.504125 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.504074    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a"} err="failed to get container status \"7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a\": rpc error: code = NotFound desc = could not find container \"7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a\": container with ID starting with 7a128063a1a09d94c7dcab453383d8f1f8422ef4edc4789d5276cc54d445f87a not found: ID does not exist"
Aug 04 21:38:49.504125 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.504093    2563 scope.go:117] "RemoveContainer" containerID="749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710"
Aug 04 21:38:49.512560 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.512545    2563 scope.go:117] "RemoveContainer" containerID="749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710"
Aug 04 21:38:49.512806 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:38:49.512787    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710\": container with ID starting with 749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710 not found: ID does not exist" containerID="749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710"
Aug 04 21:38:49.512862 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.512813    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710"} err="failed to get container status \"749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710\": rpc error: code = NotFound desc = could not find container \"749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710\": container with ID starting with 749f44565e9ccccd15fbd8bd2ad65788dc9dfa3840dffa6cb3483fd8298cf710 not found: ID does not exist"
Aug 04 21:38:49.517425 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.517405    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"]
Aug 04 21:38:49.521877 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.521856    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-80be56-5fc6498ccb-w2rtn"]
Aug 04 21:38:49.535516 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.535496    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"]
Aug 04 21:38:49.541258 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:49.541237    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-80be56-7d67896d64-fgdj4"]
Aug 04 21:38:51.244245 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:51.244211    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99c04223-3aed-47f8-adac-ea1a48416d03" path="/var/lib/kubelet/pods/99c04223-3aed-47f8-adac-ea1a48416d03/volumes"
Aug 04 21:38:51.244782 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:38:51.244758    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b03620ff-baed-43b5-8d82-15349d8020eb" path="/var/lib/kubelet/pods/b03620ff-baed-43b5-8d82-15349d8020eb/volumes"
Aug 04 21:39:03.041067 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.041037    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"]
Aug 04 21:39:03.041661 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.041283    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79" podUID="24b760e1-3098-4eae-9680-de234ca82947" containerName="maas-api" containerID="cri-o://25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a" gracePeriod=30
Aug 04 21:39:03.289652 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.289625    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:39:03.432476 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.432445    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq7cw\" (UniqueName: \"kubernetes.io/projected/24b760e1-3098-4eae-9680-de234ca82947-kube-api-access-mq7cw\") pod \"24b760e1-3098-4eae-9680-de234ca82947\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") "
Aug 04 21:39:03.432687 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.432492    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24b760e1-3098-4eae-9680-de234ca82947-openshift-service-ca\") pod \"24b760e1-3098-4eae-9680-de234ca82947\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") "
Aug 04 21:39:03.432687 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.432545    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls\") pod \"24b760e1-3098-4eae-9680-de234ca82947\" (UID: \"24b760e1-3098-4eae-9680-de234ca82947\") "
Aug 04 21:39:03.433010 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.432958    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24b760e1-3098-4eae-9680-de234ca82947-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "24b760e1-3098-4eae-9680-de234ca82947" (UID: "24b760e1-3098-4eae-9680-de234ca82947"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:39:03.435024 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.434994    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24b760e1-3098-4eae-9680-de234ca82947-kube-api-access-mq7cw" (OuterVolumeSpecName: "kube-api-access-mq7cw") pod "24b760e1-3098-4eae-9680-de234ca82947" (UID: "24b760e1-3098-4eae-9680-de234ca82947"). InnerVolumeSpecName "kube-api-access-mq7cw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:39:03.435142 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.435062    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "24b760e1-3098-4eae-9680-de234ca82947" (UID: "24b760e1-3098-4eae-9680-de234ca82947"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:39:03.533493 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.533460    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq7cw\" (UniqueName: \"kubernetes.io/projected/24b760e1-3098-4eae-9680-de234ca82947-kube-api-access-mq7cw\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:39:03.533493 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.533487    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24b760e1-3098-4eae-9680-de234ca82947-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:39:03.533493 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.533496    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24b760e1-3098-4eae-9680-de234ca82947-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:39:03.575672 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.575629    2563 generic.go:356] "Generic (PLEG): container finished" podID="24b760e1-3098-4eae-9680-de234ca82947" containerID="25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a" exitCode=0
Aug 04 21:39:03.575672 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.575672    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79" event={"ID":"24b760e1-3098-4eae-9680-de234ca82947","Type":"ContainerDied","Data":"25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a"}
Aug 04 21:39:03.575926 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.575723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79" event={"ID":"24b760e1-3098-4eae-9680-de234ca82947","Type":"ContainerDied","Data":"a9d0db01207cc7f96ef2339b641863c64ec21153f9b9b2cf108ffa97a510104a"}
Aug 04 21:39:03.575926 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.575678    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"
Aug 04 21:39:03.575926 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.575749    2563 scope.go:117] "RemoveContainer" containerID="25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a"
Aug 04 21:39:03.585296 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.585275    2563 scope.go:117] "RemoveContainer" containerID="25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a"
Aug 04 21:39:03.585606 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:39:03.585583    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a\": container with ID starting with 25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a not found: ID does not exist" containerID="25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a"
Aug 04 21:39:03.585705 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.585611    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a"} err="failed to get container status \"25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a\": rpc error: code = NotFound desc = could not find container \"25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a\": container with ID starting with 25052a6b9456a28b2e53b78498372b45daf534a8781b76a94e083882b0315e3a not found: ID does not exist"
Aug 04 21:39:03.600800 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.600762    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"]
Aug 04 21:39:03.603194 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:03.603171    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a189ec-75dc8f8784-8kp79"]
Aug 04 21:39:05.243017 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:05.242983    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24b760e1-3098-4eae-9680-de234ca82947" path="/var/lib/kubelet/pods/24b760e1-3098-4eae-9680-de234ca82947/volumes"
Aug 04 21:39:16.923863 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:16.923768    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"]
Aug 04 21:39:16.924375 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:16.924021    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn" podUID="57d3d212-b9ce-40da-ac77-367f135fce62" containerName="maas-api" containerID="cri-o://40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008" gracePeriod=30
Aug 04 21:39:17.180703 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.180648    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:39:17.250150 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.250125    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57d3d212-b9ce-40da-ac77-367f135fce62-openshift-service-ca\") pod \"57d3d212-b9ce-40da-ac77-367f135fce62\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") "
Aug 04 21:39:17.250330 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.250163    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcl7q\" (UniqueName: \"kubernetes.io/projected/57d3d212-b9ce-40da-ac77-367f135fce62-kube-api-access-gcl7q\") pod \"57d3d212-b9ce-40da-ac77-367f135fce62\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") "
Aug 04 21:39:17.250330 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.250191    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57d3d212-b9ce-40da-ac77-367f135fce62-maas-api-tls\") pod \"57d3d212-b9ce-40da-ac77-367f135fce62\" (UID: \"57d3d212-b9ce-40da-ac77-367f135fce62\") "
Aug 04 21:39:17.250470 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.250447    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57d3d212-b9ce-40da-ac77-367f135fce62-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57d3d212-b9ce-40da-ac77-367f135fce62" (UID: "57d3d212-b9ce-40da-ac77-367f135fce62"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 21:39:17.252341 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.252291    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57d3d212-b9ce-40da-ac77-367f135fce62-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57d3d212-b9ce-40da-ac77-367f135fce62" (UID: "57d3d212-b9ce-40da-ac77-367f135fce62"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 21:39:17.252471 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.252454    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57d3d212-b9ce-40da-ac77-367f135fce62-kube-api-access-gcl7q" (OuterVolumeSpecName: "kube-api-access-gcl7q") pod "57d3d212-b9ce-40da-ac77-367f135fce62" (UID: "57d3d212-b9ce-40da-ac77-367f135fce62"). InnerVolumeSpecName "kube-api-access-gcl7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 21:39:17.351695 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.351661    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57d3d212-b9ce-40da-ac77-367f135fce62-openshift-service-ca\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:39:17.351695 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.351689    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcl7q\" (UniqueName: \"kubernetes.io/projected/57d3d212-b9ce-40da-ac77-367f135fce62-kube-api-access-gcl7q\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:39:17.351695 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.351698    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57d3d212-b9ce-40da-ac77-367f135fce62-maas-api-tls\") on node \"ip-10-0-129-249.ec2.internal\" DevicePath \"\""
Aug 04 21:39:17.656260 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.656227    2563 generic.go:356] "Generic (PLEG): container finished" podID="57d3d212-b9ce-40da-ac77-367f135fce62" containerID="40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008" exitCode=0
Aug 04 21:39:17.656454 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.656272    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"
Aug 04 21:39:17.656454 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.656274    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn" event={"ID":"57d3d212-b9ce-40da-ac77-367f135fce62","Type":"ContainerDied","Data":"40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008"}
Aug 04 21:39:17.656454 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.656363    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn" event={"ID":"57d3d212-b9ce-40da-ac77-367f135fce62","Type":"ContainerDied","Data":"d60a557400e92994a23648c580f8e6ae7b1fdfa415ebbe906b5be64fd231d3d4"}
Aug 04 21:39:17.656454 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.656396    2563 scope.go:117] "RemoveContainer" containerID="40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008"
Aug 04 21:39:17.665913 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.665892    2563 scope.go:117] "RemoveContainer" containerID="40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008"
Aug 04 21:39:17.666179 ip-10-0-129-249 kubenswrapper[2563]: E0804 21:39:17.666162    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008\": container with ID starting with 40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008 not found: ID does not exist" containerID="40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008"
Aug 04 21:39:17.666230 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.666186    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008"} err="failed to get container status \"40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008\": rpc error: code = NotFound desc = could not find container \"40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008\": container with ID starting with 40c310accfabdb5787905cef4b72f9335603413cdd7cdf73531fa5f4a85a3008 not found: ID does not exist"
Aug 04 21:39:17.682376 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.682349    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"]
Aug 04 21:39:17.687333 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:17.687293    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cd644a-5b5769bb84-pbghn"]
Aug 04 21:39:19.243821 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:19.243778    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57d3d212-b9ce-40da-ac77-367f135fce62" path="/var/lib/kubelet/pods/57d3d212-b9ce-40da-ac77-367f135fce62/volumes"
Aug 04 21:39:25.162078 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:25.162047    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c5c755469-flqx5_a25d6c26-82d0-4529-a211-97fc6ba88095/authorino/0.log"
Aug 04 21:39:30.485925 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:30.485896    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5dcrr_cad6b73e-db31-49f4-a41d-02c538737b4d/manager/0.log"
Aug 04 21:39:30.605116 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:30.605087    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rnkjr_4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a/manager/0.log"
Aug 04 21:39:30.729046 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:30.729021    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f8b8cf5-fzrmj_629e6a83-3e5e-4e36-894a-b2b72bb04ccc/manager/0.log"
Aug 04 21:39:30.841240 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:30.841172    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bf984_3c796804-5024-4777-a3f3-ad7efdd8e45c/server/0.log"
Aug 04 21:39:30.960216 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:30.960183    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nxf79_ef35ac4c-7176-42bc-9cee-39ada1daab47/manager/0.log"
Aug 04 21:39:31.076131 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:31.076106    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6889885b4f-5bstj_e797d989-4a7d-4f6d-8494-1b1cd9a56292/copy-manifests/0.log"
Aug 04 21:39:31.088878 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:31.088853    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6889885b4f-5bstj_e797d989-4a7d-4f6d-8494-1b1cd9a56292/manager/0.log"
Aug 04 21:39:31.686479 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:31.686453    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6cf454fbc-wxq2w_24eb8796-719a-4fbb-aae5-56c09a42bff7/maas-api/0.log"
Aug 04 21:39:32.152972 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:32.152945    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-phwdq_c2104bfe-57ee-4590-90c9-4ddda089ba85/postgres/0.log"
Aug 04 21:39:32.917259 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:32.917231    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c5c755469-flqx5_a25d6c26-82d0-4529-a211-97fc6ba88095/authorino/0.log"
Aug 04 21:39:33.036566 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:33.036541    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-jfjfl_03a04462-0175-400e-9968-f5f71987f040/manager/0.log"
Aug 04 21:39:33.146697 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:33.146671    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-w2lr5_cfbc6f6f-3db4-4083-9ca2-3814c9124571/manager/0.log"
Aug 04 21:39:33.371637 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:33.371608    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-q4fjn_72f0c04c-30f8-4f74-9848-142de69eca43/manager/0.log"
Aug 04 21:39:33.604260 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:33.604228    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-r52cb_a8caded2-8d9a-4c3f-a37f-93295d41ef4f/manager/0.log"
Aug 04 21:39:34.361227 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:34.361198    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-586b55cbcc-v8lvb_9ad7142d-85b9-4a89-980c-f9367032d5d8/kube-auth-proxy/0.log"
Aug 04 21:39:34.601409 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:34.601379    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-822qv_4a8647a6-432a-4079-830f-ef18a945607b/payload-pre-processing/0.log"
Aug 04 21:39:34.835880 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:34.835851    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6765f794fb-9thrr_6bea9a8c-2700-49e3-8d87-3456d6d2c3e9/router/0.log"
Aug 04 21:39:35.296473 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:35.296450    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-l8dgj_d0587a57-26e0-4696-93af-54049fabc2e3/main/0.log"
Aug 04 21:39:35.635661 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:35.635636    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6pvp9_42e1d088-a11f-40ba-a522-04437b8a7f34/main/0.log"
Aug 04 21:39:35.748899 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:35.748873    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-k74rf_4829dc1f-2116-4de9-9472-be6f00f7ec5b/main/0.log"
Aug 04 21:39:43.659205 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:43.659179    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-62hnz_6a2ac5f5-4e0f-4ac4-92cf-2281241d4e33/global-pull-secret-syncer/0.log"
Aug 04 21:39:43.898751 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:43.898726    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vh5t2_7876b090-ff49-42b7-aff9-93b9aa25ef12/konnectivity-agent/0.log"
Aug 04 21:39:43.923075 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:43.923012    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-249.ec2.internal_f3d43927280795fd50a92f8253826fd4/haproxy/0.log"
Aug 04 21:39:48.929894 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:48.929862    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c5c755469-flqx5_a25d6c26-82d0-4529-a211-97fc6ba88095/authorino/0.log"
Aug 04 21:39:48.972450 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:48.972427    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-jfjfl_03a04462-0175-400e-9968-f5f71987f040/manager/0.log"
Aug 04 21:39:49.020570 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:49.020549    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-w2lr5_cfbc6f6f-3db4-4083-9ca2-3814c9124571/manager/0.log"
Aug 04 21:39:49.146941 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:49.146911    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-q4fjn_72f0c04c-30f8-4f74-9848-142de69eca43/manager/0.log"
Aug 04 21:39:49.356981 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:49.356912    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-r52cb_a8caded2-8d9a-4c3f-a37f-93295d41ef4f/manager/0.log"
Aug 04 21:39:50.982770 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:50.982744    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-7cwsh_3dbdac82-ac1f-49c9-9f64-98c6f63e8076/cluster-monitoring-operator/0.log"
Aug 04 21:39:51.124538 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:51.124512    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-86cd44d5cf-qzt5j_581f4a82-3bde-43ab-a599-ab8198976690/metrics-server/0.log"
Aug 04 21:39:51.364163 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:51.364135    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zf4cn_586b8879-a1db-4d3c-b342-5ea6cef440f8/node-exporter/0.log"
Aug 04 21:39:51.392058 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:51.391993    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zf4cn_586b8879-a1db-4d3c-b342-5ea6cef440f8/kube-rbac-proxy/0.log"
Aug 04 21:39:51.417453 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:51.417434    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zf4cn_586b8879-a1db-4d3c-b342-5ea6cef440f8/init-textfile/0.log"
Aug 04 21:39:53.956624 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.956551    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"]
Aug 04 21:39:53.957104 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957001    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b03620ff-baed-43b5-8d82-15349d8020eb" containerName="maas-api"
Aug 04 21:39:53.957104 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957017    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b03620ff-baed-43b5-8d82-15349d8020eb" containerName="maas-api"
Aug 04 21:39:53.957104 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957031    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed576e8-991f-41d7-8615-76ebb86a0e8d" containerName="maas-api"
Aug 04 21:39:53.957104 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957040    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed576e8-991f-41d7-8615-76ebb86a0e8d" containerName="maas-api"
Aug 04 21:39:53.957104 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957069    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b760e1-3098-4eae-9680-de234ca82947" containerName="maas-api"
Aug 04 21:39:53.957104 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957077    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b760e1-3098-4eae-9680-de234ca82947" containerName="maas-api"
Aug 04 21:39:53.957104 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957085    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99c04223-3aed-47f8-adac-ea1a48416d03" containerName="payload-processing"
Aug 04 21:39:53.957104 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957093    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="99c04223-3aed-47f8-adac-ea1a48416d03" containerName="payload-processing"
Aug 04 21:39:53.957104 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957106    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe7e60b0-1657-4fea-8532-5f3a1d5e03f2" containerName="maas-api"
Aug 04 21:39:53.957484 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957114    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe7e60b0-1657-4fea-8532-5f3a1d5e03f2" containerName="maas-api"
Aug 04 21:39:53.957484 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957130    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57d3d212-b9ce-40da-ac77-367f135fce62" containerName="maas-api"
Aug 04 21:39:53.957484 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957137    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="57d3d212-b9ce-40da-ac77-367f135fce62" containerName="maas-api"
Aug 04 21:39:53.957484 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957219    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed576e8-991f-41d7-8615-76ebb86a0e8d" containerName="maas-api"
Aug 04 21:39:53.957484 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957230    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b03620ff-baed-43b5-8d82-15349d8020eb" containerName="maas-api"
Aug 04 21:39:53.957484 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957239    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe7e60b0-1657-4fea-8532-5f3a1d5e03f2" containerName="maas-api"
Aug 04 21:39:53.957484 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957247    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b760e1-3098-4eae-9680-de234ca82947" containerName="maas-api"
Aug 04 21:39:53.957484 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957256    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="57d3d212-b9ce-40da-ac77-367f135fce62" containerName="maas-api"
Aug 04 21:39:53.957484 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.957263    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="99c04223-3aed-47f8-adac-ea1a48416d03" containerName="payload-processing"
Aug 04 21:39:53.961295 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.961278    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:53.963766 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.963747    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nzqjh\"/\"kube-root-ca.crt\""
Aug 04 21:39:53.964176 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.964157    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nzqjh\"/\"default-dockercfg-cpcnx\""
Aug 04 21:39:53.964858 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.964842    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nzqjh\"/\"openshift-service-ca.crt\""
Aug 04 21:39:53.970122 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:53.970100    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"]
Aug 04 21:39:54.062608 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.062564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-sys\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.062782 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.062677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-podres\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.062782 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.062704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxxp4\" (UniqueName: \"kubernetes.io/projected/34013083-08b1-445e-91ac-d71a53d5ba37-kube-api-access-cxxp4\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.062874 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.062778    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-proc\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.062874 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.062808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-lib-modules\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.163254 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.163226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-proc\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.163254 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.163255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-lib-modules\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.163474 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.163288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-sys\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.163474 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.163376    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-proc\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.163474 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.163391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-podres\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.163474 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.163415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxxp4\" (UniqueName: \"kubernetes.io/projected/34013083-08b1-445e-91ac-d71a53d5ba37-kube-api-access-cxxp4\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.163474 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.163433    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-lib-modules\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.163474 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.163447    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-sys\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.163698 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.163505    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34013083-08b1-445e-91ac-d71a53d5ba37-podres\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.172279 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.172256    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxxp4\" (UniqueName: \"kubernetes.io/projected/34013083-08b1-445e-91ac-d71a53d5ba37-kube-api-access-cxxp4\") pod \"perf-node-gather-daemonset-chq2f\" (UID: \"34013083-08b1-445e-91ac-d71a53d5ba37\") " pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.271920 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.271848    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.403034 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.403009    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"]
Aug 04 21:39:54.404804 ip-10-0-129-249 kubenswrapper[2563]: W0804 21:39:54.404775    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod34013083_08b1_445e_91ac_d71a53d5ba37.slice/crio-f061cd5eb22759941bcee0f89ba76f9743373d9cc78f14d990f77f630dc5761f WatchSource:0}: Error finding container f061cd5eb22759941bcee0f89ba76f9743373d9cc78f14d990f77f630dc5761f: Status 404 returned error can't find the container with id f061cd5eb22759941bcee0f89ba76f9743373d9cc78f14d990f77f630dc5761f
Aug 04 21:39:54.864240 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.864209    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f" event={"ID":"34013083-08b1-445e-91ac-d71a53d5ba37","Type":"ContainerStarted","Data":"29ce6fc8b910cfb38ef69c00cb3aa18b55d9478fd2eaed0701275868d31e95ee"}
Aug 04 21:39:54.864416 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.864269    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f" event={"ID":"34013083-08b1-445e-91ac-d71a53d5ba37","Type":"ContainerStarted","Data":"f061cd5eb22759941bcee0f89ba76f9743373d9cc78f14d990f77f630dc5761f"}
Aug 04 21:39:54.864416 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.864373    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:39:54.883670 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:54.883629    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f" podStartSLOduration=1.883614637 podStartE2EDuration="1.883614637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 21:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 21:39:54.882302991 +0000 UTC m=+3542.125922273" watchObservedRunningTime="2026-08-04 21:39:54.883614637 +0000 UTC m=+3542.127233919"
Aug 04 21:39:56.498032 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:56.498003    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f9bch_fea94516-a7b5-41cc-84c5-10a37f3d021a/dns/0.log"
Aug 04 21:39:56.524478 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:56.524455    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f9bch_fea94516-a7b5-41cc-84c5-10a37f3d021a/kube-rbac-proxy/0.log"
Aug 04 21:39:56.689079 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:56.689054    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kblfw_62b003b3-9dd3-4f30-b0ac-c88b3003e065/dns-node-resolver/0.log"
Aug 04 21:39:57.282630 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:57.282601    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c6f664b95-wkksp_cdeb039d-a981-49dc-bb7c-fe1f126f75b7/registry/0.log"
Aug 04 21:39:57.416330 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:57.416289    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w7fs7_f558bf91-5880-4917-9e1e-c78ae5493458/node-ca/0.log"
Aug 04 21:39:58.788279 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:58.788248    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-586b55cbcc-v8lvb_9ad7142d-85b9-4a89-980c-f9367032d5d8/kube-auth-proxy/0.log"
Aug 04 21:39:58.867478 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:58.867452    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-822qv_4a8647a6-432a-4079-830f-ef18a945607b/payload-pre-processing/0.log"
Aug 04 21:39:58.925815 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:58.925793    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6765f794fb-9thrr_6bea9a8c-2700-49e3-8d87-3456d6d2c3e9/router/0.log"
Aug 04 21:39:59.777280 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:39:59.777256    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fg7km_3142a84a-6e2c-4b9d-8d2b-a86223d81428/serve-healthcheck-canary/0.log"
Aug 04 21:40:00.351838 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:00.351804    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f872z_db41f9ff-f898-4cda-b1b0-2bc2dba136ba/kube-rbac-proxy/0.log"
Aug 04 21:40:00.375091 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:00.375066    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f872z_db41f9ff-f898-4cda-b1b0-2bc2dba136ba/exporter/0.log"
Aug 04 21:40:00.398407 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:00.398386    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f872z_db41f9ff-f898-4cda-b1b0-2bc2dba136ba/extractor/0.log"
Aug 04 21:40:00.880088 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:00.880059    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nzqjh/perf-node-gather-daemonset-chq2f"
Aug 04 21:40:02.840610 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:02.840529    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5dcrr_cad6b73e-db31-49f4-a41d-02c538737b4d/manager/0.log"
Aug 04 21:40:03.048222 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:03.048186    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rnkjr_4d4e8b5c-2b50-46eb-9cbb-06318cd33d1a/manager/0.log"
Aug 04 21:40:03.232168 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:03.232136    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f8b8cf5-fzrmj_629e6a83-3e5e-4e36-894a-b2b72bb04ccc/manager/0.log"
Aug 04 21:40:03.269361 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:03.269309    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bf984_3c796804-5024-4777-a3f3-ad7efdd8e45c/server/0.log"
Aug 04 21:40:03.342717 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:03.342687    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nxf79_ef35ac4c-7176-42bc-9cee-39ada1daab47/manager/0.log"
Aug 04 21:40:03.436031 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:03.435993    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6889885b4f-5bstj_e797d989-4a7d-4f6d-8494-1b1cd9a56292/manager/0.log"
Aug 04 21:40:03.462513 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:03.462488    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6889885b4f-5bstj_e797d989-4a7d-4f6d-8494-1b1cd9a56292/copy-manifests/0.log"
Aug 04 21:40:04.884484 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:04.884460    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-l22tj_ce7c5b5f-e412-4e4b-bf21-c4007b9395d1/openshift-lws-operator/0.log"
Aug 04 21:40:04.898387 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:04.898367    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-586b55cbcc-v8lvb_9ad7142d-85b9-4a89-980c-f9367032d5d8/kube-auth-proxy/0.log"
Aug 04 21:40:04.959265 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:04.959243    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-822qv_4a8647a6-432a-4079-830f-ef18a945607b/payload-pre-processing/0.log"
Aug 04 21:40:05.001998 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:05.001979    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6765f794fb-9thrr_6bea9a8c-2700-49e3-8d87-3456d6d2c3e9/router/0.log"
Aug 04 21:40:06.791586 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:06.791523    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-586b55cbcc-v8lvb_9ad7142d-85b9-4a89-980c-f9367032d5d8/kube-auth-proxy/0.log"
Aug 04 21:40:06.859309 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:06.859279    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-822qv_4a8647a6-432a-4079-830f-ef18a945607b/payload-pre-processing/0.log"
Aug 04 21:40:06.900095 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:06.900057    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6765f794fb-9thrr_6bea9a8c-2700-49e3-8d87-3456d6d2c3e9/router/0.log"
Aug 04 21:40:10.046778 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:10.046746    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-ckq4n_00bac56d-cfc6-441d-9a61-126ff8c60642/migrator/0.log"
Aug 04 21:40:10.069967 ip-10-0-129-249 kubenswrapper[2563]: I0804 21:40:10.069944    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-ckq4n_00bac56d-cfc6-441d-9a61-126ff8c60642/graceful-termination/0.log"